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Preface 



Audience: As teachers and students of Combinatorial Optimization, we have often looked 
for material that illustrates the elegance of classical results on matchings, trees, matroids 
and flows while at the same time highlights methods that have continued application. With 
the advent of approximation algorithms, some techniques from exact optimization such as 
the primal-dual method have indeed proven their staying power and versatility. In this 
book, we describe what we believe is a simple and powerful method that is iterative in 
essence, and useful in a variety of settings. 

The core of the iterative methods we describe relies on a fundamental result in linear 
algebra that the row rank and column rank of a real matrix are equal. This seemingly 
elementary fact allows us via a counting argument to provide an alternate proof of the 
above-mentioned classical results; the method is constructive and the resulting algorithms 
are iterative with the correctness proven by induction. Furthermore, these methods gener- 
alize to accommodate a variety of additional constraints on these classical problems that 
render them NP-hard - a careful adaptation of the iterative method leads to very effective 
approximation algorithms for these cases. 

Our goal in this book has been to highlight the commonality and uses of this method 
and convince the readers of the generality and potential for future applications. We have 
used an elementary presentation style that should be accessible to anyone with introductory 
college mathematics exposure in linear algebra and basic graph theory. Whatever advanced 
material in these areas we require, we develop from scratch along the way. Some basic 
background on approximation algorithms such as is provided in the various books and 
surveys available on this subject will be useful in appreciating the power of the results we 
prove in this area. Other than the basic definition of an approximation algorithm and the 
understanding of polynomial-time complexity, no further technical background is required 
from this typically more advanced subject. 

An important secondary goal of the book is to provide a framework and material 
for introductory courses in combinatorial optimization at the upper-class undergraduate 
and beginning graduate levels. We hope the common approach across the chapters gives a 
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comprehensive way to introduce these topics for the first time. The more advanced applica- 
tions are useful illustrations for graduate students of their potential for future application 
in their research. 

History: This book is inspired by the application of the iterative method in the field 
of approximation algorithms and its recent adaptations to prove performance guarantees 
for problems with two objectives. This adaptation showed us how the proof technique 
can be used to re-prove several classical results in combinatorial optimization and also 
in approximation algorithms in a unified way. The book owes its origin to the paper by 
Jain [91] describing a 2-approximation algorithm for a large class of minimum-cost network- 
design problems in undirected networks. While there are other earlier illustrations of the 
method in the literature, it is Jain's work that inspired the adaptation that led to the 
results in this monograph. 

Jain's result itself was a breakthrough when it appeared, and demonstrated the 
power of his iterative rounding method to prove this result that was conjectured based on 
a long line of earlier papers that applied a different primal-dual method to these problems. 
In this sense, his method was a purely primal attack on the problem. His method was 
extended by Lau, Naor, Salavatipour and Singh [110] to degree-bounded network design 
problems. The adaptation of this method by Singh and Lau [154] to the degree-bounded 
minimum-cost spanning tree problem surprisingly involves no rounding at all! Instead, 
variables whose value are set to one in the linear programming relaxation are selected and 
the program is modified carefully to continue to yield this property. This explains the title 
of this monograph and also hints at how this adaptation now allows one to prove exact 
results since we no longer have to round any variables and lose optimality. 

Acknowledgments: We are grateful to the many organizations whose support 
have enabled this work: US National Science Foundation, Research Grants Council of 
Hong Kong, Microsoft Research, Kyoto University RIMS, the Qatar Foundation, Carnegie 
Mellon University - Pittsburgh and Doha, McGill University and the Chinese University 
of Hong Kong. We are also grateful to our families for their support of this endeavor. We 
hope you will enjoy reading this monograph as much as we did writing it. 

Dedications: Lau dedicates this work to his parents, his wife Pui Ming and their 
children Ching Lam, Sing Chit and Ching Yiu. Ravi dedicates this work to the memory 
of his late brother, R. Balasubramaniam, who encouraged him to write a book. Singh 
dedicates this work to his parents. 
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1 

Introduction 



f In this first chapter we motivate our method via the assignment problem. Through 
this problem we highlight the basic ingredients and ideas of the method. We then give 
an outline of how a typical chapter in the rest of the book is structured, and how the 
remaining chapters are organized. 



1.1 The Assignment Problem 

Consider the classical assignment problem: Given a bipartite graph G = (Vi U V^,E) with 
|^i| = |^2| and weight function w : E — ^ R + , the objective is to match every vertex in V\ 
with a distinct vertex in V2 to minimize the total weight (cost) of the matching. This is 
also called the minimum weight bipartite perfect matching problem in the literature, and 
is a fundamental problem in combinatorial optimization. See Figure 1.1 for an example of 
a perfect matching in a bipartite graph. 




Fig. 1.1. The solid edges form a perfect matching in the bipartite graph. 

t © Copyright 2011 by Lap Chi Lau, R. Ravi and Mohit Singh. Published 2011 by Cambridge 
University Press. 



One approach to the assignment problem is to model it as a linear programming 
problem. A linear program is a mathematical formulation of the problem with a system 
of linear constraints which can contain both equalities and inequalities, and also a linear 
objective function that is to be maximized or minimized. In the assignment problem, we 
associate a variable x uv for every {u, v} £ E. Ideally, we would like the variables to take 
one of two values, zero or one (hence in the ideal case, they are binary variables). When 
x uv is set to one, we intend the model to signal that this pair is matched; when x uv is set 
to zero, we intend the model to signal that this pair is not matched. The following is a 
linear programming formulation of the assignment problem. 



minimize w uv x uv 

u,v 

subject to 'y y x uv = 

v:{u,v}gE 

x uv = I Vt) € V 2 

u:{u,v}eE 

x uv > \/{u,v} G E 



The objective function is to minimize the total weight of the matching, while the two sets 
of linear equalities ensure that every vertex in V\ is matched to exactly one vertex in V2 
in the assignment and vice-versa. 

A fundamental result in the Operations Research literature [86] is the polynomial 
time solvability (as well as the practical tractability) of linear programming problems. 
There is also a rich theory of optimality (and certificates for it) that has been developed 
(see e.g., the text by Chvatal [36]). Using these results, we can solve the problem we have 
formulated above quite effectively for even very large problem sizes. 

Returning to the formulation however, our goal is to find a "binary" assignment of 
vertices in V\ to vertices in V2, but in the solution returned, the rc-variables may take 
fractional values. Nevertheless, for the assignment problem, a celebrated result that is 
a cornerstone of combinatorial optimization [37] states that for any set of weights that 
permit a finite optimal solution, there is always an optimal solution to the above LP 
(linear program) that takes binary values in all the x-variables. 

Such integrality results of LPs are few and far between, but reveal rich underlying 
structure for efficient optimization over the large combinatorial solution space [150]. They 
have been shown using special properties of the constraint matrix of the problem (such 
as total unimodularity), or of the whole linear system including the right hand side (such 
as total dual integrality). This book is about a simple and fairly intuitive method that 
is able to re-prove many (but not all) of the results obtained by these powerful methods. 
One advantage of our approach is that it can be used to incorporate additional constraints 
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that make the problem computationally hard, and allow us to derive good approximation 
algorithms with provable performance guarantee for the constrained versions. 



1.2 Iterative Algorithm 

Our method is iterative. Using the following two steps, it works inductively to show that 
the LP has an integral optimal solution. 

• If any x uv is set to 1 in an optimal solution to the LP, then we take this pair as matched 
in our solution, and delete them both to get a smaller problem, and proceed to the next 
iteration. 

• If any variable x uv is set to in an optimal solution, we remove the edge (u,v) to again 
get a smaller problem (since the number of edges reduces by 1) and proceed to the next 
iteration. 

We continue the above iterations till all variables have been fixed to either or 1. 
Given the above iterative algorithm, there are two claims that need to be proven. Firstly, 
that the algorithm works correctly, i.e., it can always find a variable with value or 1 
in each iteration and secondly, the matching selected is an optimal (minimum weight) 
matching. Assuming the first claim, the second claim can be proved by a simple inductive 
argument. The crux of the argument is that in each iteration our solution pays exactly 
what the fractional optimal solution pays. Moreover, the fractional optimal solution when 
restricted to the residual graph remains feasible for the residual problem. This allows us to 
apply an inductive argument to show that the matching we construct has the same weight 
as the fractional optimal solution, and is thus optimal. For the first claim, it is not clear 
a-priori that one can always find a variable with value 1 or at every step. Indeed the 
example in Figure 1.2 shows that there might not be such a variable at some fractional 
optimal solution. However, we use the important concept of the extreme point (or vertex) 
solutions of linear program to show that the above iterative algorithm works correctly. 

Definition 1.2.1 Let P = {x : Ax = b, x > 0} C M ra . Then x 6 M n is an extreme point 

solution of P if there does not exist a non-zero vector y 6 W 1 such that x + y, x — y G P. 

Extreme point solutions are also known as vertex solutions and are equivalent to 
basic feasible solutions. These concepts are defined in Chapter 2. Pictorially extreme 
point solutions are the corner points of the set of feasible solutions. The following basic 
result shows that there is always an optimal extreme point solution to bounded linear 
programs. 

Lemma 1.2.2 Let P = {x : Ax = b, x > 0} and assume that the optimum value min{c T x : 
x £ P} is finite. Then for any feasible solution x £ P, there exists an extreme point 
solution x' 6 P with c T x' < c T x. 
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(a) (b) (c) 

Fig. 1.2. In Figure (a), the fractional solution which places \ on all the edges is an optimal 
fractional solution but not an extreme point solution. The fractional solution in Figure (a) is the 
convex combination of the integral solutions in Figure (b) and Figure (c) . 

The following Rank Lemma is an important ingredient in the correctness proofs of 
almost all iterative algorithms in this monograph (see Chapter 2) . 

Lemma 1.2.3 (Rank Lemma) Let P = {x : Ax = b, x > 0} and let x be an extreme 
point solution of P such that x% > for each i. Then the number of variables is equal to 
the number of linearly independent constraints of A, i.e. the rank of A. 



1.2.1 Contradiction Proof Idea: Lower Bound > Upper Bound 

We give an outline of the proof that at each iteration there exists a variable with value 
or 1. Suppose for contradiction that < x uv < 1 for every edge {u, v} G E. We 
use this assumption to derive a lower bound on the number of variables of the linear 
program. Let n be the remaining vertices in V\ (or V2, they have the same cardinality) 
at the current iteration. Then each vertex in V\ must have two edges incident on it, since 
YlveV2-(u v)eE x ™> = 1 and x uv < 1 for each (u,v) G E. Thus the total number of edges is 
at least 2n. This is a lower bound on the number of variables of the linear program, since 
we have one variable for each edge. 

On the other hand, using the Rank Lemma, we derive an upper bound on the number 
of variables of the linear program. In the linear program for bipartite matching, we have 
only 2n constraints (one for each vertex in V\ U V2). Moreover, these 2n constraints 
are dependent since the sum of the constraints for vertices in V\ equals the sum of the 
constraints for vertices in V2. Hence, the number of linearly independent constraints is at 
most 2n — 1. By the Rank Lemma, the number of variables is at most In — 1. This provides 
us an upper bound on the number of variables. Since our upper bound is strictly smaller 
than the lower bound, we obtain the desired contradiction. Therefore, in an extreme point 
solution of the linear program for bipartite matching, there must exist a variable with value 
or 1, and thus the iterative algorithm works. The number of iterations can be simply 
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bounded by the number of edges in the bipartite graph. We give a formal proof of the 
above outline in Chapter 3. 

1.2.2 Approximation Algorithms for NP-hard Problems 

The above framework can be naturally adapted to provide an approximation algorithm via 
the iterative method. In particular, for this, the iterative algorithm above typically has 
one or both of two additional steps: Rounding and Relaxation. 

(i) Rounding: Fix a threshold a > 1. If there is a variable Xi that has a value of at 
least ^ in the optimal extreme point solution then pick the corresponding element 
in the solution being constructed. 

(ii) Relaxation: Fix a threshold (3. If there is a constraint Yli a i x i ^ such that 
Yli a i — b + P then remove the constraint in the residual formulation. 

For the bipartite matching problem, we will see how the iterative algorithm presented 
above can be adapted to give approximation algorithms for the generalized assignment 
problem in Chapter 3. Other generalizations include the budgeted allocation problem in 
Chapter 3 and the hypergraph matching problem in Chapter 9. 

1.3 Approach Outline 

We now give an overview of the structure of the rest of the monograph. Early chapters in 
the book contain two main components: the first deals with proving the integrality of the 
LP relaxation of a well-studied problem, while the second shows how the iterative proof 
of integrality can be extended to design approximation algorithms for NP-hard variants of 
these basic problems. Both components follow the natural outline described below. 

(i) Linear Programming Formulation: We start by giving a linear programming 
relaxation for the optimization problem we study. If the problem is polynomially 
solvable, this relaxation will be one with integral extreme points and that is what 
we will set out to show. If the problem is NP-hard, we state an approximation 
algorithmic result which we then set out to prove. 

(a) Solvability: Sometimes the linear programming relaxation we start with 
will be exponential in size. We then show that the linear program is solv- 
able in polynomial time. Usually, this would entail providing a polynomial 
time separation oracle for the program using the formalism of the ellipsoid 
method [81]. Informally, the separation oracle is a procedure that certifies 
that any given candidate solution for the program is either feasible or not, 
and in the latter case provides a separating hyperplane which is a violated in- 
equality of the formulation. In programs with an exponential number of such 
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inequalities that are implicity described, the design of the separation oracle 
is itself a combinatorial optimization problem, and we sketch the reduction 
to one. 

(ii) Characterization of Extreme Point Solution: We then give a characterization 
result for the optimal extreme point solutions of the linear program based on the 
Rank Lemma (Lemma 1.2.3). This part aims to show that any maximal set of 
linearly independent tight constraints at this extreme point solution can be captured 
by a sparse structure. Sometimes the proof of this requires the use of the uncrossing 
technique [37] in combinatorial optimization, which will be introduced in Chapter 4. 

(iii) Iterative Algorithm: We present an iterative algorithm for constructing an in- 
tegral solution to the problem from an extreme point solution. The algorithm has 
two simple steps. 

(a) If there is a variable in the optimal extreme point solution that is set to a 
value of 1, then include the element in the integral solution. 

(b) If there is a variable in the optimal extreme point solution that is set to a 
value of 0, then remove the corresponding element. 

In each of the above cases, at each iteration, we reduce the problem and 
arrive at a residual version, then we recompute an optimal extreme point solution 
and iterate the above steps until all variables have been set this way. In designing 
approximation algorithms we also use the rounding and relaxation steps as stated 
earlier. 

(iv) Analysis: We then analyze the iterative algorithm. This involves arguing the 
following two facts. First, we establish that the algorithm runs correctly and second, 
that it returns an optimal solution. 

(a) Correctness: We show that the iterative algorithm is correct by arguing 
that there is always an 1-element or a 0-element to pick in every iteration. 
This crucially uses the characterization of tight constraints at this optimal 
extreme point solution. The argument here also follows the same contradic- 
tion proof idea (lower bound > upper bound) : We assume for a contradiction 
that there is no 1-element or 0-element and get a large lower bound on the 
number of nonzero variables in the optimal extreme point solution. On the 
other side, we use the sparsity of the linearly independent tight constraints 
to show an upper bound on the number of such constraints. This then con- 
tradicts the Rank Lemma that insists that both these numbers are equal, 
and proves that there is always an 1- or 0-element. 

(b) Optimality: We finally show that the iterative algorithm indeed returns 
an optimal solution using a simple inductive argument. The crux of this 
argument is to show that the extreme point solution induced on the residual 
problem remains a feasible solution to this residual problem. 
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For the NP-hard variants of the problems we study, our goal is to show that the 
above framework can be naturally adapted to provide an approximation algorithm via the 
iterative method. In particular, recall that the iterative algorithm above typically has one 
or both of two additional steps: Rounding and Relaxation. 

(i) Rounding: Fix a threshold a > 1. If there is a variable Xj which in the optimal 
extreme point solution has a value of at least ^ then include the corresponding 
element in the solution. 

Adding this rounding step does not allow us to obtain optimal integral solution 
but only near-optimal solutions. Using the above step, typically one obtains an 
approximation ratio of a for covering problems addressed using this framework. 

(ii) Relaxation: Fix a threshold /3. If there is a constraint Yl% a i x i ^ such that 
Yli a i — b + P then remove the constraint in the residual formulation. 

The iterative relaxation step removes a constraint and hence this constraint 
can be violated in later iterations. But the condition on the removal of the con- 
straints ensures that the constraint is only violated by an additive amount of j3. 
This step enables us to obtain additive approximation algorithms for a variety of 
problems. 

To summarize, for designing approximation algorithms, we first study the exact op- 
timization problem in the above framework. We then use the above two steps in various 
combinations to derive strong approximation algorithms for constrained versions of these 
exact problems. In the last few chapters we find a few examples of approximation algo- 
rithms that do not strictly fit this framework (e.g. multicriteria versions, cut problems, 
bin packing) but the overall approach for these problems remains the same. 

1.4 Context and Applications of Iterative Rounding 

One goal in presenting the collections of results in this book is to convince the reader that 
iterative rounding is an effective tool in proving results in optimization. As with any tool, 
a key question is when is this tool applicable and what are the alternates? 

The iterative method for exact optimization used a rank-based argument of the 
sparsity of the solution to argue integrality of a proposed linear programming formulation 
of the underlying problem. In Section 13.2, we detail the earliest application we know of 
this method to prove Steinitz's result on rearrangements. 

As we mentioned in the introduction, the iterative method for approximation al- 
gorithms was introduced in the work of Jain on the survivable network design problem. 
For this minimum-cost subgraph selection problem, Jain formulated a covering linear pro- 
gram and showed how any extreme point always has a positive variable of value at least 
half; he did this by using the sparsity of the extreme point solution, that followed from 
a rank-based argument. In this context, the iterative method is a specific version of the 
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deterministic rounding paradigm applied to LP relaxations for NP-hard problems. Thus 
it fits in the broader context of a variety of other LP rounding methods for the design of 
approximation algorithms including randomized rounding, primal-dual methods and La- 
grangean relaxations. Among these methods, iterative rounding is particularly applicable 
in solving multi-objective problems where a base problem is complicated by more than one 
objective function: Examples include the bipartite matching problem complicated by addi- 
tional load constraints at each node to give the NP-hard Generalized Assignment problem, 
or the minimum spanning tree problem complicated by degree constraints on nodes gives 
the NP-hard bounded-degree MST problem. An understanding of the iterative method 
applied to the base problem is then a useful guide to extending its application to the 
constrained multi-objective versions. 

1.5 Book Chapters Overview 

In the next chapter, we develop all the preliminaries needed in the following chapters. 
We discuss linear programs, and their polynomial time solvability using the separation 
oracle. We also outline the important Rank Lemma and other properties about extreme 
point solutions. We also discuss the LP duality theorem and the complementary slackness 
conditions, and some basic facts about submodular functions and graphs. 

A first stream of chapters study problems in undirected graphs. In Chapter 3, we give 
the first example to illustrate the iterative method on bipartite matching and vertex cover 
problems. We also show how the proof for the bipartite matching leads to approximation 
algorithms for the generalized assignment problem and the budgeted allocation problem. In 
Chapter 4, we study the classical spanning tree problem and its extension to the minimum 
bounded degree spanning tree problem. This chapter introduces the uncrossing technique 
in combinatorial optimization. In Chapter 5, we generalize the arguments for undirected 
spanning trees to bases of matroids as well as to the common bases in the intersection of 
two matroids, and also to the minimum bounded degree matroid basis problem and the 
maximum common independent set problem in the intersection of k matroids. We also 
show integrality of the dual of matroid and matroid intersection problems which lead to 
certain min-max results. 

A second stream of chapters study problems in directed graphs. In Chapter 6, we 
study the directed rooted spanning tree (or arborescence) problem, along with a degree- 
bounded version, and then generalize the method developed here to a rooted A;-connected 
subgraph problem providing a self-contained proof of a result of Frank and Tardos [58]. This 
is developed further in Chapter 7 to showing the integrality of submodular flow problems. 
For this last problem, we again complement the proof of exact LP characterization with a 
description of an approximation algorithm for the degree-bounded version built upon the 
proof of the exact counterpart. For the submodular flow problem, we also give a proof of 
the integrality of its dual. 

We then present a few more advanced chapters applying the iterative method. In 



15 



Chapter 8, we apply the iterative method to general problems involving network matrices 
as constraint matrices (with integral right hand sides) and their duals. We then show 
the application of network matrices to derive integrality of the duals of various linear pro- 
grams encountered in earlier chapters (such as those for matroid bases, matroid intersection 
and submodular flow). In Chapter 9, we address the generalization of perfect and maxi- 
mum matchings in bipartite graphs to general graphs, and also address higher dimensional 
matching problems. We then present a common generalization of Jain's 2-approximation 
algorithm for the survivable network design problem (SNDP), and a result of Boyd and 
Pulleyblank on 1-edges in the Held-Karp relaxation for the Symmetric Traveling Sales- 
man Problem (STSP) in Chapter 10. This chapter also generalizes Jain's result to degree 
bounded network design problems. In Chapter 11, we extend the application of the method 
to constrained optimization problems such as partial covering and multicriteria problems. 
In Chapter 12, we add the primal-dual complementary slackness conditions to the iterative 
method to derive approximation results for some cut problems. In Chapter 13 we present 
some early examples of iterative methods, including the Beck-Fiala theorem on discrepancy 
and Karmarkar-Karp algorithm for bin packing. Most chapters contain selected historical 
notes as well as exercises. 



1.6 Notes 

Polyhedral combinatorics, the compact polyhedral description of important combinatorial 
optimization problems, is a fundamental and unifying tool in algorithms, combinatorics 
and optimization. A highlight of this line of research is the pioneering work by Jack 
Edmonds [44]; we refer the reader to the book [150] and the historical survey [148] by 
Schrijver for an encyclopedic treatment of this subject. 

Two closely related methods for proving integrality of polyhedra that are widely 
covered in Schrijver's book deserve mention: Total Unimodularity (TU) and Total Dual 
Integrality (TDI). Informally, TU matrices are constraint matrices such that for integral 
right hand sides, the linear programming relaxations provide integral solutions (whenever 
the solutions exist and are finite). Alternately, using the relation between extreme points 
solutions and basic feasible solutions to LPs developed in the next chapter, these matrices 
are those for which every square submatrix has determinant value zero, plus one or minus 
one. The class of Network matrices that we will study in Chapter 8 is an important example 
of such TU matrices. Total Dual Integrality involves both the constraint matrix and the 
right hand side: a system of inequalities defined by a constraint matrix and right hand 
side vector is TDI if for all integer objective coefficients, the dual program has an integral 
solution (whenever it exists and is finite). If a system is TDI for an integral right hand 
side, then the polyhedon described by the system is integral hence giving another way of 
providing characterizations of integral solutions to combinatorial optimization problems. 
A popular example of an integral characterization that arises from a TDI system is the 
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description of matchings in general graphs that we develop using our alternate iterative 
method in Chapter 9. 

An implicit use of the iterative method is found in the alternate proof of Steinitz's 
theorem due to Grinberg and Sevastyanov [156, 79, 12]. Earlier uses of the iterative re- 
laxation method can be traced back to the proof of a discrepancy theorem by Beck and 
Fiala [22] and the approximation algorithm for the bin packing problem by Karmarkar 
and Karp [93]. In approximation algorithms, the first explicit use of the iterative rounding 
method is due to Jain [91]. 

An iterative approach similar to the one we describe is used in bounding quantities 
of interest in randomly chosen combinatorial objects and is termed the "semi random 
method". For more details, see the books by Alon and Spencer [2] or Molloy and Reed [122]. 
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2 

Preliminaries 



f In this chapter we discuss linear programming and basic facts about extreme point so- 
lutions to linear programs. We then briefly discuss solution methods for linear programs, 
particularly stating the sufficiency of finding a separation oracle for the program to be able 
to solve it. We then state some concepts from graph theory which are used throughout 
the book. The last part of the chapter discusses submodular and supermodular func- 
tions. These functions give a general tool for modeling a variety of optimization problems. 
Excellent introductory textbooks or surveys in all three areas are available for further 
reference [20, 89, 160]. 



2.1 Linear Programming 

Using matrix notation, a linear program is expressed as follows. 

... t 
minimize c x 

subject to Ax > b 

x > 

If x satisfies (Ax > b, x > 0), then x is a feasible solution. If there exists a feasible 
solution to the linear program, it is feasible; otherwise it is infeasible. An optimal solution 
x* is a feasible solution such that c T x* = min{c T a; s.t. Ax > b, x > 0}. The linear program 
is unbounded (from below) if VA G R, 3 feasible x such that c T x < A. 

There are different forms in which a linear program can be represented. However, 
all these forms are equivalent to the form we consider above and can be converted into one 
another by simple linear transformations (see e.g., [36]). 

t © Copyright 2011 by Lap Chi Lau, R. Ravi and Mohit Singh. Published 2011 by Cambridge 
University Press. 
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2.1.1 Extreme Point Solutions to Linear Programs 

In this subsection, we discuss basic properties about extreme point solutions (Defini- 
tion 1.2.1). First, we have the following definition. 

Definition 2.1.1 Let P be a polytope and let x be an extreme point solution of P then 
x is integral if each co-ordinate of x is an integer. The polytope P is called integral if 
every extreme point of P is integral. 

We now show basic properties about extreme point (or vertex) solutions. Most proofs 
are quite standard and we give a short sketch. The reader is referred to standard texts on 
Linear Programming (e.g. Chvatal [36]) for details. We now prove Lemma 1.2.2. We state 
it again for completeness. 

Lemma 2.1.2 Let P = {x : Ax > b, x > 0} and assume that min{c T x : x G P} is finite. 
Then for every there exists an extreme point solution x' G P such that c T x' < c T x, 

i.e., there is always an extreme point optimal solution. 

Proof The idea of the proof is to show by how we can move from a current optimal solution 
to one that has more zero components or more tight constraints and is thus closer to being 
an extreme point solution. 

Consider x such that it is optimal but not an extreme point solution. That implies 
there exists y ^ such that x + y G P and x — y G P. Therefore, 

A{x + y) > b, x + y>0 
A(x - y) > b, x -y > 

Let A = be the submatrix of A restricted to rows which are at equality at x, and b = 
be the vector b restricted to these rows. Thus we have A = x = b = . Hence, we must have 
A = y > and A = (—y) > 0. Subtracting, we get A = y = 0. Since x is optimal, the following 
holds. 

c T x < c T [x + y) 
c T x < c T (x — y) 
=>■ c T y = 

Moreover, since y / 0, without loss of generality assume there exists j such that 
yj < (if not then use — y) . Consider x + Xy for A > and increase A until x + Ay is no 
longer feasible due to the non-negativity constraints on x. Formally, let 

x * . t . Xj . AiX bi 1 
A = mm{ mm — — , mm ) 

j:yj<0 —yj i-.AiX^^AiyKO —Aiy 

We now show that x + \*y is a new optimal solution with one more zero coordinate or 
one extra tight constraint. Since x + y > and x — y > 0, if Xi = then yi = 0. Therefore, 
the coordinates that were at 0, remain at 0. Moreover A = (x+y) = A = x = b since A = y = 0, 
hence tight constraints remain tight. Since we assume that min{c T x : x G P} is finite, A* 



19 



is finite and the solution x + X*y has one more zero coordinate (when A* = (xj)/(—yj)) or 
one extra tight constraint (when A* = {AiX — bi) / {— Am)) . 

Proceeding this way, we can convert any optimal solution to one that is also an 
extreme point solution, proving the claim. □ 

The next theorem relates extreme point solutions to corresponding non-singular 
columns of the constraint matrix. 

Lemma 2.1.3 Let P = {x : Ax > b, x > 0}. For x 6 P, let A = be the submatrix of 
A restricted to rows which are at equality at x, and let A^ denote the submatrix of A = 
consisting of the columns corresponding to the nonzeros in x. Then x is an extreme point 
solution if and only if A^ has linearly independent columns (i.e., A= has full column rank). 

Proof (•<=) If x is not an extreme point solution, we will show that A= has linearly 
dependent columns. By the hypothesis, there exists y / such that A = y = (see the proof 
of the previous theorem). Therefore A= (the columns where y has a nonzero coordinate) 
has linearly dependent columns. By the observation made at the end of the previous proof, 
x • = =>- yj = 0. Therefore, A= is a submatrix of A=. Therefore, the columns of A= are 
linearly dependent. 

(=>) We want to show that if A= has linearly dependent columns then x is not 
an extreme point solution. By the hypothesis, there exists y ^ such that A=y = 0. 
Complete y to an n-dimensional vector by setting the remaining coordinates to 0. Now 
by construction, A = y = 0. Moreover, by construction y,j = whenever Xj = 0. Note that 
there exists e > such that x + ey > and x — ey > 0. Also x + ey and x — ey are feasible 
since A{x + ey) = Ax + eAy > b and A{x — ey) > b for small enough e > 0. Hence, x is 
not an extreme point solution. □ 

We now prove the important Rank Lemma. We restate the Lemma (in canonical 
form) for completeness. 

Lemma 2.1.4 (Rank Lemma) Let P = {x : Ax > b, x > 0} and let x be an extreme 
point solution of P such that x% > for each i. Then any maximal number of linearly 
independent tight constraints of the form A^x = b{ for some row i of A equals the number 
of variables. 

Proof Since X{ > for each i, we have A= = A = . From Lemma 2.1.3 it follows that A = 
has full column rank. Since the number of columns equals the number of non-zero variables 
in x and row rank of any matrix equals the column rankf , we have that row rank of A = 
equals the number of variables. Then any maximal number of linearly independent tight 
constraints is exactly the maximal number of linearly independent rows of A = which is 
exactly the row rank of A = and hence the claim follows. □ 

f Important check: If you are rusty on why this statement is true, a crisp proof of the equality of the row rank and 
column rank can be found in the short note due to Andrea and Wong [4] that is available on the web. 
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Next, we highlight various methods of solving linear programs. First we introduce the 
concept of basic feasible solutions and show their equivalence to extreme point solutions. 
Basic feasible solutions form a key ingredient in the Simplex algorithm which is the most 
widely used algorithm for solving linear programs in practice. 

2.1.1.1 Basic Feasible Solution 

Consider the linear program 

... t 
minimize c x 

subject to Ax > b 

x > 

By introducing slack variables Sj for each constraint, we obtain an equivalent linear 
program in standard form. 

... t 
minimize c x 

subject to Ax + s = b 

x > 

s > 

Henceforth, we study linear program in standard form: {mincx : Ax = b, x > 0}. 
Without loss of generality, we can assume that A is of full row rank. If there are dependent 
constraints, we can remove them without affecting the system or its optimal solution. 

A subset of columns B of the constraint matrix A is called a basis if the matrix of 
columns corresponding to B, i.e. Ab, is invertible. A solution x is called basic if and only 
if there is a basis B such that Xj = if j '• £ B and xb = A^b. If in addition to being 
basic, it is also feasible, i.e., A~j^b > 0, it is called a basic feasible solution for short. The 
correspondence between bases and basic feasible solutions is not one to one. Indeed there 
can be many bases which correspond to the same basic feasible solution. The next theorem 
shows the equivalence of extreme point solutions and basic feasible solutions. 

Theorem 2.1.5 Let A be a m x n matrix with full row rank. Then every feasible x to 
P = {x : Ax = b, x > 0} is a basic feasible solution if and only if x is an extreme point 
solution. 

Proof (=>) If x is a basic feasible solution, then Ab is invertible. Since A x is a submatrix 
of Ab (it is a proper submatrix if some basic variable is at 0), A x has linearly independent 
columns. Therefore, by Lemma 2.1.3, x is an extreme point solution. 

(<=) If x is an extreme point solution, then by Lemma 2.1.3, A x has linearly inde- 
pendent columns. Now we can add columns to A x from A to convert it into an invertible 
matrix Ab- Note that since Ax = b, AbXb + Aj^xn = b, where An and xn denote the 
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non-basic parts of A and x respectively. By construction of As, xjy = and so xb = A B b. 
So x is a basic feasible solution with As as the basis. □ 



2.1.2 Algorithms for Linear Programming 

The simplex algorithm solves linear programs to get a basic feasible optimal solution. 
It works by starting at any basic feasible solution and moving to a neighboring basic 
feasible solution which improves the objective function. The convexity of the linear program 
ensures that once the simplex algorithm ends at a local optimum basic feasible point, it 
has achieved the global optimum as well. Many variants of the simplex algorithm have 
been considered, each defined by which neighboring basic feasible solution to move in case 
there are more than one improving basic feasible points in the neighborhood. Although 
the simplex algorithm works efficiently in practice, there are examples where each variant 
of the simplex algorithm runs in exponential time. Again, for more details, see e.g. [36]. 

Polynomial-time algorithms for solving linear programs fall in two categories: ellip- 
soid algorithms [95] and interior point algorithms [92]. We refer the reader to Nemhauser 
and Wolsey [130] and Wright [164] for details about these algorithms. Both these algo- 
rithms solve linear programs to obtain near optimal solution in polynomial time. Moreover, 
there are rounding algorithms [130] which, given a sufficiently near optimal solution to a 
linear program, return an optimal extreme point solution. 

Theorem 2.1.6 There is an algorithm which returns an optimal extreme point solution to 
a linear program. Moreover, the running time of the algorithm is polynomial in the size of 
the linear program. 

2.1.3 Separation and Optimization 

In this book, we will also encounter linear programs where the number of constraints is 
exponential in the size of the problem (e.g., in the spanning tree problem in Chapter 4, 
we will write linear programs where the number of constraints is exponential in the size 
of the graph) and it is not obvious that one can enumerate them, let alone solve them in 
polynomial time. We use the notion of separation to show that many exponentially sized 
linear programs can be solved in polynomial time. 

Definition 2.1.7 Given x* 6 1Z n and a polytope P = {x : Ax > b, x > 0}, the separation 

problem is the decision problem whether x* £ P. The solution of the separation problem 
is the answer to the membership problem and in case x* ^ P, it should return a valid 
constraint AiX > b{ for P which is violated by x* , i.e., AiX* < b{. 

The following theorem of Grotschel, Lovasz and Schrijver [81] shows that polynomial 
time separability is equivalent to polynomial time solvability of a linear program; we state 
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it in a form that is convenient for combinatorial optimization problems. The basis of this 
equivalence is the ellipsoid algorithm. 

Theorem 2.1.8 Given a full- dimensional polytope P and a polynomial-time separation 
oracle for P, one can find an optimal extreme point solution to a linear objective function 
over P (assuming it is bounded) via the Ellipsoid algorithm that uses a polynomial number 
of operations and calls to the separation oracle. 

Clearly, one can solve the separation problem by checking each constraint but for 
problems where the number of constraints is exponential in size such a method is too slow. 
In this book, as we consider LP formulations with an exponential number of constraints, 
we will often provide efficient separation oracles showing that the linear program for the 
problem is solvable in polynomial time. 

2.1.4 Linear Programming Duality 

Linear programming duality is a key concept to certify and characterize optimal solutions 
to linear programs. Consider the following primal linear program in the standard form: 



V 1 < i < m 
V 1 < j < n 



V 1 < j < n 
V 1 < i < m 

It is not difficult to show that the optimal value of the primal linear program is at least 
the optimal value of the dual linear program, and thus any dual feasible solution provides 
a lower bound on the optimal value of the primal program. This is called the weak LP 
duality theorem, whose proof also follows from the derivation of the complementary slack- 
ness conditions below. A fundamental result in linear programming is the strong duality 
theorem, which shows that the optimal value of the primal linear program is actually equal 
to that of the dual linear program. 

Theorem 2.1.9 (Strong Duality Theorem) If the primal linear program has an optimal 
solution, so does its dual, and the respective optimal costs are equal. 



11 

minimize ^ CjXj 

i=i 

n 

subject to 

> h, 

3=1 

Xj > 



The corresponding dual program is 



maximize biyi 
i=i 

m 

subject to '^^aiji/i < Cj 

i=i 

Vi > 
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Many combinatorial min-max theorems can be derived from the strong duality the- 
orem. For example, we will see in Chapter 3 the min-max theorem for bipartite matching, 
and in Chapter 5 for the min-max theorem for matroid intersection. We refer the reader to 
any textbook on linear programming (e.g. [36]) for the proof of the strong duality theorem. 

2.1.4-1 Complementary Slackness Conditions 

The complementary slackness conditions provide a characterization for an optimal primal 
solution x and an optimal dual solution y. We will use the complementary slackness 
conditions in Chapter 12. 

Primal complementary slackness conditions: 



Either Xj — or 



= 1 



Dual complementary slackness conditions: 

n 

Either yi = or ^ a ij x j = 

3 = 1 

These conditions can be derived as follows: 

n n m 

j=i j=i i=i 

m n 

i=i j=i 



> 



=i 



where the first inequality is by the constraints in the dual linear program, the second 
equality is by interchanging the order of the summations, and the third inequality is by 
the constraints in the primal linear program. Note that this shows the weak duality the- 
orem. Since x and y are optimal solutions, by the strong duality theorem, we have that 
YTj=\ c j x j = Yl^iLibiVii an d thus equality must hold throughout. The primal comple- 
mentary slackness conditions follow from the first inequality holding as an equality, while 
the dual complementary slackness conditions follow from the last inequality holding as an 
equality. 



2.2 Graphs and Digraphs 

Most problems addressed in this book are on networks connecting nodes with edges or 
links. We define graph theoretic concepts which will be encountered in later chapters. 
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Given an undirected graph G = (V,E) and a set S C V, we denote Sg(S) or Se(S) be the 
set of edges which have exactly one endpoint in S. For a vertex v G V, 5g({v}) is simply 
denoted by 5q(v). We also denote dc{v) or <1e(v) to be degree of t>, i.e., |<5g(v)|. For sets 
X, Y C V, we denote Eq(X : Y) to be the set of edges which has exactly one endpoint in X 
and one in Y. We also denote E G (X, X) by £ G P0- Observe that 5 G {X) = E G (X,V\X). 
We also denote |<5g(X)| by da(X). The subscript G or E 1 is sometimes dropped from the 
notation if the graph G is clear from the context. A subgraph H of G is spanning if it 
contains all the vertices of G, connected if there is a path between any two vertices of H, 
a tree if it is acyclic and connected. An important concept is a spanning tree, subgraph 
which is both spanning and a tree. Observe that a spanning tree is a also a minimally 
spanning connected subgraph. 

Given a directed graph D = (V, A) and a set S C V, we denote 8 l p(S) to be the set 
of arcs whose head is in S but tail is not in S. Similarly, 5^(5) is the set of arcs whose 
tail is in S but the head is not in S. For a vertex v G V, we denote 5 % £({v}) as 5™(v) and 
^L)' t ({ v }) as ^E) 1 ^)- The in-degree of v, \5 % ^(v)\ is denoted by dj}(v) and the out-degree 
of v, I^xj 1 *^)! is denoted by dffifo). The degree of v, do{v) is the sum of its in-degree and 
out-degree. For sets X, Y C V, we denote Eu(X,Y) to be the set of arcs whose tail is 
in X and the head is in Y. Observe that 5° I ? t (X) = E D (X,V\ X). We denote \6%(X)\ 
and ^^(X)] by d l g(X) and d°^; t {X) respectively. A subgraph H of D is called strongly 
connected if there is a directed path from each vertex of H to every other vertex, weakly 
connected if the underlying undirected graph is connected, acyclic if there is no directed 
cycle in H. H is called an arborescence if the underlying graph is a spanning tree and 
the graph has only one vertex with no in-edges which is called the root. If the root of an 
arborescence is the vertex r then it is also called an r-arborescence. 

Let e = {u,v} be an edge. By G/e we denote the graph obtained from G by 
contracting u,v into a single vertex x, while keeping all the edges in 5q({u,v}) (an edge in 
G with an endpoint in {u, v} becomes an edge in G/e with an endpoint in x) and removing 
the edges between u and v. Contracting an arc uv in a directed graph D is defined similarly, 
while an arc in D with head/tail in {u,v} becomes an arc in D/e with head/tail in x. 

We will use frequently the following fundamental min-max theorem in graph theory. 



Theorem 2.2.1 (Menger's Theorem [121]) Let D = (V,A) be a directed graph, and 
s, t 6 V be two distinct vertices. The maximum number of arc-disjoint s-t paths in D is 
equal to the minimum d l £(X) over all X C V with s ^ X and t G X. 

Menger's theorem shows a close connection between disjoint paths and cuts, which will be 
used in writing linear programs and constructing separation oracles. One can also obtain 
the corresponding min-max theorems for undirected graphs and for vertex connectivity by 
simple transformations (see the exercises). 
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2.3 Submodular and Supermodular Functions 

In this section, we define special classes of set functions with some nice convexity-like 
properties. Typically, in our applications, these functions are defined over a set of vertices 
of a graph we will be working with; most of the time, they will also be integer valued and 
positive. More comprehensive treatments on these topics are available in the monograph 
by Fujishige [62] and the book by Schrijver [150]. 

2.3.1 Submodularity 

Definition 2.3.1 A function f : 2 V — >■ R is submodular if for every pair A,B of subsets 
of V , we have 

f(A) + f(B)>f(AnB) + f(AUB). 

A simple example of a submodular set function defined on the vertices of an undi- 
rected graph G = (V, E) is cardinality of the cut function d : 2 V — > Z + where d(S) = \S(S)\. 

Proposition 2.3.2 The cut function d of any undirected graph is submodular. 

Proof To see that d is submodular, note that on the right hand side, we have 

d{A nB) = \E{A n B, A \ B)\ + \E(A n B, B \ A)\ + \E(A n B, V \ (A U B))\. 



A 



Fig. 2.1. In this example the solid edges are counted exactly once in both the LHS d(A) + d(B), 
and the RHS d(A C\B) + d(A U B), and the bold edge is counted exactly twice on both sides. The 
dashed edge is counted in LHS but not in RHS. 

Similarly we also have 

d(A U B) = \E(A n B, V \ {A U B))\ + \E(A \ B, V \ (A U B))\ + \E(B \A,V\(AUB))\. 
On the left hand side we have 

d(A) = \E(AnB,V\(AuB))\ + \E(AnB,B\A)\ + \E(A\B,V\(AuB))\ + \E(A\B,B\A)\. 
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Similarly we get 

d{B) = \E(AnB,V\(AuB))\ + \E(A(lB, A\B)\ + \E(B\A, V\(AUB))\ + \E(B\A,A\B)\. 

Comparing the above expressions shows that the edges in E(A \B, B\A) are responsible 
for the inequality (rather than equality). Also see Figure 2.1. □ 

Note that the edge cut function can be extended to the case when nonnegative 
weights, say x : E — >■ IR+ are assigned to the edges. Instead of d(S) = \5(S)\, we have 
x(8(S)) = J2ee5(S) x e- The above proof also shows that the function x is submodular as 
well. 

Proposition 2.3.3 The weighted-cut function of any undirected graph is submodular. 

For undirected graphs, since the degree function is symmetric (i.e. d(S) = d(V — S)), 
by applying Proposition 2.3.2 on the complements we have 

d(A) + d(B) > d(A \B) + d(B \ A), (2.1) 

which can also be verified directly using the same method as above; see Figure 2.2. 




Fig. 2.2. In this example the solid edges are counted exactly once in both the LHS d(A) + d(B), 
and the RHS d(A \B) + d(B \ A), and the bold edge is counted exactly twice on both sides. The 
dashed edge, however is counted in the LHS but not in the RHS. 

Let us define a stronger notion of submodularity. 

Definition 2.3.4 A function f : 2 V — > R is strongly submodular if for every pair A,B of 
subsets of V , we have 

f(A) + f(B)>f(AnB) + f(AuB) 

and 

f(A) + f(B)>f(A\B) + f(B\A). 

The second property in the definition above has also been referred to as posi- 
modularity [125, 126]. The edge cut functions in undirected graphs are strongly submodu- 
lar. Indeed, if a function is submodular and symmetric then it is strongly submodular (see 
exercises) . 
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2.3.2 Supermodularity 



We move on to a symmetric concept, supermodularity. 

Definition 2.3.5 A function f : 2 V — > M. is supermodular if for every pair A, B of subsets 
of V , we have 

f(A) + f(B)<f(AnB) + f(AuB). 

As before, a simple example of a supermodular set function denned on the vertices 
of an undirected graph G = (V, E) is the induced edge function d : 2 V — > Z + where i(S) 
is the number of edges in E with both endpoints in S, i.e., i(S) = \E(S)\. A verification 
similar to the above can be carried out to establish that the induced edge function is 
supermodular. Also, if nonnegative values, say x : E — > M + are assigned to the edges and 
we consider x(S) = J2eeE(S) x e, it follows that this function is supermodular as well. 

Proposition 2.3.6 The induced edge function for any undirected graph is supermod- 
ular. This is also true for the weighted version with nonnegative weights. 

2.3.3 Refinements 

Definition 2.3.7 Two subsets A and B of a ground set V are intersecting if Af] B / 0, 
A — B / and B — A / 0. A function f : 2 V — > R is intersecting-submodular if for every 
pair A, B of intersecting subsets of V , we have 

f(A) + f(B)>f(AnB) + f(AuB). 

Definition 2.3.8 Two subsets A and B of a ground set V are crossing if none of the 
four subsets A(~) B,A \ B,B \ A and V \ (A U B) are empty. A function f : 2 V -)• R is 
crossing-submodular if for every pair A, B of crossing subsets of V , we have 

f(A) + f(B)>f(AnB) + f(AuB). 

To distinguish the regular submodular functions from the more restricted intersecting 
and crossing varieties, they are also sometimes dubbed fully submodular. Other important 
examples of fully submodular functions arises as cut functions of directed graphs D = 
(V,A). Define 5 in (S) for a subset S of vertices as the set of arcs whose heads are in S and 
tails are in V \ S, which we can denote as A(V \ S,S). Symmetrically, define 6 out (S) as 
the set A(S,V\S). Denote \5 in (S)\ and \5 out (S)\ by d in {S) and d out (S) respectively. Both 
these functions d m and d out defined on vertex subsets are fully submodular; see Figure 2.3. 
Unlike undirected graphs, however, the functions d m and d out are not strongly submodular. 

Proposition 2.3.9 The cut functions d m and d out of any directed graph are submodular. 
This is also true for the weighted directed cut functions with non-negative weights. 
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Fig. 2.3. In this example the solid arcs are counted exactly once in both the LHS S out (A) + S out (B) 
and the RHS 5 out (Au B) + 5 out (An B) and the bold edges are counted exactly twice on both sides. 
The dashed edges are counted in the LHS but not in the RHS. 

A broader class of functions generalizing supermodularity is useful in specifying con- 
nectivity requirements for network design problems. 

Definition 2.3.10 A function f : 2 V — > M. is skew (or weakly) supermodular if for every 
pair A, B of subsets of V , at least one of the following inequalities is true. 

f(A) + f(B)<f(AnB) + f{AUB) 
f(A) + f(B)<f(A\B) + f(B\A) 

In the survivable network design problem on an undirected graph G = (V,E), we 
are given nonnegative integral edge-connectivity requirements r uv for all pairs of vertices, 
and we are interested in finding a subgraph with at least r uv edge-disjoint paths between 
u and v for every pair u, v. If we define the connectivity requirement function / for any 
set S as f(S) = fnax u ^g v as r uv , it is not hard to verify that / is skew supermodular. 

Functions that are both submodular and supermoduar are modular - the typical 
example being the cardinality (or "modulus") function. Furthermore, / is submodular if 
and only if — / is supermodular, and if / is submodular and g is supermodular, then / — g 
is submodular. Finally, if / is skew supermodular and g is strongly submodular, then f — g 
is skew supermodular. These follow directly as a consequence of the definitions. 

2.3.3.1 Minimizing submodular function 

A rich literature examines the minimization of submodular functions in (strongly) poly- 
nomial time - Chapter 45 of the book by Schrijver [150] contains most references on this 
topic. We mention three important results in this vein: Queyranne [136] gave an algorithm 
for minimizing symmetric submodular functions, building on earlier work of Nagamochi 
and Ibaraki for finding a minimum cut in an undirected graph. Grotschel, Lovasz and 
Schrijver [82] gave a strongly polynomial time algorithm for finding the minimum of a 
submodular function only over sets of odd cardinality, building on earlier work of Pad- 
berg and Rao [133]. Finally, for the general case of submodular function with only oracle 
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access to values, strongly polynomial time algorithms were first presented by Iwata, Fleis- 
cher and Fujishige [90] and also independently by Schrijver [147]. More generally defined 
submodular function (e.g., on intersecting or cross-free families) can also be minimized in 
polynomial time - see Chapter 49 in [150]. 

We will be making use of submodular (and supermodular) functions extensively 
in this monograph. In a typical application, the set of tight constraints at an extreme 
point solution corresponds to a set of cuts in a graph for which a (submodular) function 
value is an integer. Depending on the type of submodular function (full, intersecting or 
crossing), these tight constraints can then be uncrossed to result in a nicely structured 
basis for them: For instances, for fully submodular constraints, the basis is a chain; for 
intersecting submodular constraint forms, the basis is a laminar family representable by 
the forest capturing set-inclusion relation; for crossing submodular constraint systems, the 
basis is a cross-free family which can also be turned into a laminar family. Such structured 
sparse representations of the tight constraints at any extreme point solution are the key to 
proving integrality for many problems - they pave the way to show the upper bound part 
of the general argument outlined in the beginning of this chapter, typically by a counting 
argument that is carried out inductively on a representation of these sparse families. 

Exercises 

2.1 Consider a bounded linear program. Prove that an LP solution x is a basic feasible 
solution (or extreme point solution) if and only if there is an objective function c 
such that x is the unique optimum solution. 

2.2 Prove Menger's theorem for undirected graphs: The maximum number of edge- 
disjoint s-t paths is equal to the minimum d(X) over all sets X C V with s ^ X 
and t G X. (Hint: "bidirect" the edges and apply Theorem 2.2.1.) 

2.3 Two directed s-t paths Pi and P2 are internally vertex-disjoint if V{P\) fl V(P2) = 
{s,t}. A vertex set U C V is an s-t cut if there is no directed s-t paths in 
G — U. Prove Menger's theorem for vertex connectivity: The maximum number 
of internally vertex-disjoint s-t paths is equal to the minimum size of a vertex s-t 
cut. (Hint: "split" each vertex appropriately and apply Theorem 2.2.1.) 

2.4 Derive a corresponding Menger's theorem for vertex connectivity in undirected 
graphs. 

2.5 Show that if a function is submodular and symmetric then it is strongly sub- 
modular. Hence, derive that a cut function of an undirected graph is strongly 
submodular. 

2.6 Verify Proposition 2.3.9 that the in- and out-cut functions d m and cP n ' for a digraph 
are indeed submodular but not strongly submodular. 

2.7 Verify Proposition 2.3.6. Is the induced edges (arcs) function supermodular for 
digraphs? 
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2.8 Show that a function / : 2 V — > R + is (fully) submodular if and only if f{X + v) — 
f(X) > f(Y + v)- f(Y) whenever X C Y and v £ V - X. 

2.9 Use the above equivalent definition of submodular function to derive Proposi- 
tion 2.3.2 and Proposition 2.3.9. 

2.10 Show that the connectivity requirement function for the survivable network design 
problem, f(S) = max u€ s,v(£S r uv for all S C V is skew supermodular. 

2.11 Show that the connectivity requirement function for the fc-connected subgraph 
problem, f(S) = k for every non-empty S C V is crossing supermodular. 

2.12 Show that the connectivity requirement function for the rooted-connectivity prob- 
lem, f(S) = k for every non-empty S C V with r ^ S for a specified vertex r is 
intersecting supermodular. 



31 



3 

Matching and Vertex Cover in Bipartite Graphs 



t 

In this chapter we consider two very closely related problems, maximum weighted 
matching and minimum cost vertex cover in bipartite graphs. Linear programming duality 
plays a crucial role in understanding the relationship between these problems. We will 
show that the natural linear programming relaxations for both the matching problem and 
the vertex cover problem are integral, and then use duality to obtain a min-max relation 
between them. Nevertheless, our proofs of integrality use the iterative method by arguing 
the existence of 1-elements in an extreme point solution. 

In the first section, we show the integrality of the more standard maximization version 
of the matching problem. In the following sections, we show two applications of the proof 
technique for integrality to derive approximation results for NP-hard problems. We first 
present a new proof of an approximation result for the generalized assignment problem, 
and then present an approximation result for the budgeted allocation problem. The proofs 
of both of these results develop on the integrality result for the bipartite matching problem 
and introduce the iterative relaxation method. Following this, we discuss the integrality 
of the bipartite vertex cover problem formulation and conclude with a short section on the 
duality relation between these problems and some historical notes. 



3.1 Matchings in Bipartite Graphs 

In this section, we show that the matching polytope in bipartite graphs is integral. Given 
a bipartite graph G = (V\ U V2, E) and a weight function w : E — > TZ, the maximum 
matching problem is to find a set of vertex-disjoint edges of maximum total weight. 

t © Copyright 2011 by Lap Chi Lau, R. Ravi and Mohit Singh. Published 2011 by Cambridge 
University Press. 
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3.1.1 Linear Programming Relaxation 

The linear programming relaxation for the bipartite matching problem is given by the 
following LP bm (G). 



< 1 Vu€VtUF 2 
> Ve G E 

Observe that the linear program LPi mi {G) is compact, i.e., the number of constraints 
and variables is polynomially bounded in the size of the problem. Hence, the linear program 
can be solved optimally in polynomial time using Theorem 2.1.6. 

We prove the following theorem by an iterative algorithm in the next section. 

Theorem 3.1.1 Given any weight function w there exists an integral matching M such 
that w(M) > w ■ x where x is an optimal solution to LPb m (G). 

Our proof of Theorem 3.1.1 as a corollary implies the following theorem. 
Theorem 3.1.2 The linear programming formulation LPb m (G) is integral. 



3.1.2 Characterization of Extreme Point Solutions 

Before we prove Theorem 3.1.1 we give a characterization of extreme point solutions of 
LPb m (G) for which we need a few definitions. 

For a set F C E, let x(F) denote the vector in M} E \ that has an 1 corresponding to 
each edge e 6 F, and otherwise. This vector is called the characteristic vector of F. In 
the following lemma which follows by a direct application of the Rank Lemma 2.1.4, we 
characterize an extreme point solution by a set of tight linearly independent constraints. 

Lemma 3.1.3 Given any extreme point solution x to LPb m (G) such that x e > for each 
e £ E there exists W C V\ U V2 such that 

(i) x(5(v)) = 1 for each v G W. 

(ii) The vectors in {x^M) : v ^ ^} are linearly independent. 

(iii) \W\ = \E\. 



maximize w e x e 

subject to x e 

ee6(v) 

x e 
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3.1.3 Iterative Algorithm 



We now give the algorithm which constructs an integral matching of weight at least the 
optimal solution to LPi mi (G) proving Theorem 3.1.1. The algorithm is a simple iterative 
procedure as shown in Figure 3.1. 



Iterative Bipartite Matching Algorithm 






(i) Initialization F <— 0. 






(ii) While E(G) + do 






(a) Find an optimal extreme point solution x 


to LPb m (G) and 


remove every 


edge e with x e = from G. 






(b) If there is an edge e = {u, v} with x e = 1, 


then update F <- 


- F U {e} and 


G^G\{u,v}. 






(iii) Return F. 







Fig. 3.1. Bipartite Matching Algorithm 



3.1.4 Correctness and Optimality 

We prove the correctness of the algorithm in two steps. First, we show that the algorithm 
returns a matching of optimal weight if the algorithm always finds an edge e with x e = 
in Step (ii) (a) or an edge e with x e = 1 in Step (ii) (b) . In the second part, we show that 
the algorithm will always find such an edge completing the proof. 

Claim 3.1.4 If the algorithm, in every iteration, finds an edge e with x e = in Step (ii) (a) 
or an edge e with x e = 1 in Step (ii)(b), then it returns a matching F of weight at least 
the optimal solution to LPi mi {G). 

Proof The proof will proceed by induction on the number of iterations of the algorithm. 
The base case is trivial when the algorithm proceeds for only one iteration. 

If we find an edge e with x e = in Step (ii) (a) of the algorithm, then the residual 
problem is to find a matching in the graph G' = G \ {e}, where we remove the edge 
e from G. The residual solution x res , x restricted to G \ {e}, is a feasible solution to 
the linear programming relaxation of the residual problem. By induction, the algorithm 
returns a matching F' C E{G') with weight at least the optimal solution to LP\ )m {G'). 
Since w(F') > w • x res = w • x, the induction hypothesis holds in this case. 

In the other case, if we find an edge e = {u,v} with x e = 1 in Step (ii) (b) of the 
algorithm then the residual problem is to find a matching which contains the edge e. This is 
exactly the matching problem in graph G' = G\{u, v}, where we remove the vertices u and 
v and their incident edges from G. Moreover x res , x restricted to edges in G', is a feasible 
solution to the linear programming relaxation for the residual problem. Inductively, the 
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algorithm will return a matching F' of weight at least the weight of the optimum solution 
of LP bm (G'), and hence w(F') > feasible solution to LP\ )m {G'). The 

algorithm returns the matching F = F' U {e} and we have 

w(F) = w(F') + w e and w(F') > w • x res 

which implies that 

w(F) > W ■ Xres + W e = W • X 

since x e = 1- Therefore, the weight of the matching returned by the algorithm is at least 
the weight of the LP solution x. □ 

We now complete the proof of Theorem 3.1.1 by showing that the Algorithm always 
finds an edge e with x e = or x e = 1. The proof of the following lemma crucially uses the 
characterization of extreme point solutions given in Lemma 3.1.3. 

Lemma 3.1.5 Given any extreme point solution x of LPb m (G) there must exist an edge e 
with x e = or x e = 1. 

Proof Suppose for sake of contradiction < x e < 1 for each edge e G E. Lemma 3.1.3 
implies that there exists W C I4UV2 such that constraints corresponding to W are linearly 
independent and tight, and \E\ = \ W\. 

We claim that oIe{v) = 2 for each v £ W and oIe{v) = for each v ^ W . Firstly, 
<Ie{v) > 2 for each v £ W, since x(5(v)) = 1 for each v and < x e < 1 for each e G E. 
This implies that: 

2\W\ = 2\E\ = d E {v) > ^2 d E( v ) > 2 \ w \- 
vev vew 

This implies the inequalities must hold as equalities, and thus oIe(v) = for each v ^ W 
by the first inequality and oIe(v) = 2 for each v £ W by the second inequality. 

Hence, E is a cycle cover on the vertices in W. Let C be any such cycle with all 
vertices in W. Since C is an even cycle because G is bipartite we also have 

E x(S(v))= £ X (S(v)), 
v&cnVi vecnv 2 

which contradicts the independence of constraints in condition (ii) of Lemma 3.1.3. There- 
fore any extreme point solution x to LPfe m (G) must have an edge e with x e = or x e = 1. 

□ 

Thus we obtain from Lemma 3.1.5 that the algorithm in Figure 3.1 returns a matching 
with total weight at least the weight of the linear program. This completes the proof of 
Theorem 3.1.1. 
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3.2 Generalized Assignment Problem 



In the section, we use the iterative relaxation method to obtain an approximation algorithm 
for the generalized assignment problem. The generalized assignment problem models the 
following scheduling problem on unrelated parallel machines with costs. We are given a set 
of jobs J and machines M, for each job j £ J and machine i £ M there is a processing time 
Pij and cost Cjj. Machine i is available for Tj time units and the objective is to assign each 
job to some machine such that the total cost is minimized and no machine is scheduled for 
more than its available time. 

Shmoys and Tardos [151] gave an algorithm which returns an assignment of cost at 
most C and each machine is used for at most 2Tj time units, where C is the cost of the 
optimal assignment which uses machine i for at most Tj time units (if such an assignment 
is possible). In this section, we prove the result of the Shmoys and Tardos [151] using the 
iterative relaxation method. This proof develops on the iterative proof of the integrality 
of the bipartite matching given in Section 3.1. We shall prove the following theorem. 



Theorem 3.2.1 There exists a polynomial time algorithm for the generalized assignment 
problem which returns a solution of cost at most C that uses each machine i for at most 
2Ti time units, where C is the cost of an optimal assignment that uses each machine i for 
at most Ti time units. 



3.2.1 Linear Programming Relaxation 

Before we write the linear program for the problem, we first model the problem as a 
bipartite matching problem. We start with a complete bipartite graph G with jobs J 
and machines M as the two sides of the bipartite graph. The edge between job j £ J 
and machine i £ M has cost Cjj. The generalized assignment problem can be reduced to 
finding a subgraph F of G such that (IfU) = 1 for each job j £ J and the edge incident 
at j denotes which machine job j is assigned to. The time constraint at machines can be 
modeled by restricting that ^2 e& s(i)nF Pij — for each machine i. We strengthen this 
model by disallowing certain assignments using the following observation: If > Tj then 
no optimal solution assigns job j to i, and hence we can remove all such edges from graph 
G. 

We model the above matching problem by the following natural linear programming 
relaxation LP ga to prove Theorem 3.2.1. Observe that we do not place time constraints 
for all machines but a subset M' C M which is initialized to M. We have a variable x e for 
each e = ij denoting whether job j is assigned to machine i. 
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minimize ^ Cij Xij 

e=(i,j)GE 

subject to x e = 1 Vj € J 

eG<5(j) 

VeXe < ^ Vi G M' 

e£<5(i) 

x e > Ve G -E 



3.2.2 Characterization of Extreme Point Solutions 

The following lemma follows from a direct application of the Rank Lemma. 

Lemma 3.2.2 Zei x 6e an extreme point solution to the linear program LP ga with < 
x e < 1 /or eac/i edge e. Then there exist J' C J and M" C M' smc/i i/iai 

(i) S e£ 5(j) x e = 1 /or each j G J' and I] ee5 (j)P e x e = Tj /or eac/i i G M". 

(ii) T/ie constraints corresponding to J' and M" are linearly independent. 
(hi) \J'\ + \M"\ = \E(G)\ 



3.2.3 Iterative Algorithm 

We present a simple iterative procedure which returns an assignment of optimal cost in 
Figure 3.2. Observe that the iterative procedure generalizes the iterative procedure for 
bipartite matching in Section 3.1. The bipartite graph F with vertex set in MU J returns 
the assignment found by the algorithm. 

This procedure demonstrates our first example of the iterative relaxation method 
in Step (ii) (c) . Here in addition to the usual step of picking an integral element in the 
solution, we identify carefully chosen constraints to relax or remove. The choice is dictated 
by ensuring that the removal will allow us to argue that the final integral solution does 
not have too much violation; at the same time, we need to ensure that in the absence of 
an integral element, such a constraint can always be found to be removed. The crux of 
the relaxation method is to find the right relaxation condition that balances this trade-off 
nicely. 



3.2.4 Correctness and Performance Guarantee 

The following lemma shows that the algorithm makes progress at each step of the algorithm. 
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Iterative Generalized Assignment Algorithm 

(i) Initialization E{F) <- 0, M' <- M. 

(ii) While J / do 

(a) Find an optimal extreme point solution x to LP ga and remove every vari- 
able with Xij = 0. 

(b) If there is a variable with xij — 1, then update F <- F(J{ij}, J <r- J\{j}, 
Ti 4- Ti -pij. 

(c) (Relaxation) If there is a machine i with d(i) = 1, or a machine i with 
d(i) = 2 and Y,jeJ x ii - 1 ' then u P date M' <- M' \ {i}. 

(in) Return F. 



Fig. 3.2. The Generalized Assignment Algorithm 

Lemma 3.2.3 Consider any extreme point solution x to LP ga . One of the following must 
hold. 

(i) There exists an edge e G E with x e G {0, 1}. 

(ii) There exists a machine i G M 1 with d{i) = 1, or d(i) = 2 and ^2jeJ Xi J — 1- 

Proof Suppose for sake of contradiction that both the conditions do not hold. By 
Step (ii) (a) and Step (ii)(b), we have < x e < 1 for each edge e. Each job j has de- 
gree at least two since J2 e e5(j) Xe = ^ and there is no edge with x e = 1 by Step (h)(b). 
Moreover, each machine in M' has degree at least two, because the constraints for ma- 
chines with degree one have been removed in Step (ii)(c). From Lemma 3.2.2 we have that 
\E\ = \ J'\ + \M"\. This implies that: 

\J'\ + |M"| = \E\ > EieJ d(j) + EieM ' d{i) > \J\ + \M'\ > \ J'\ + \M"\, 

and hence all inequalities must hold as equalities. The first inequality implies that each 
machine i £ M\M' has degree zero; the second inequality implies that each job j G J' and 
each machine i G M" have degree exactly two; the last inequality implies that J = J' and 
M' = M" . Therefore, G is a union of cycles, with vertices in J' U M" (tight constraints). 
Consider any cycle C. The total number of jobs in C is exactly equal to the total number 
of machines in C. Therefore, since each job j G J' has X^gM" Xi i = 1' there must be a 
machine i with YljeJ' Xi i — ^ Hence, this machine i has degree two and Ylje J' Xi i — ^> 
contradicting that Step (ii) (c) cannot be applied. □ 

We now prove Theorem 3.2.1 by a simple inductive argument. 

Proof of Theorem 3.2.1: We first prove that the algorithm returns an assignment 
of optimal cost. We claim that at any iteration of the algorithm the cost of assignment 
given by F plus the cost of the current linear programming solution to LP ga is at most the 
cost of the initial linear programming solution. This can be shown by a simple inductive 
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argument on the number of iterations. Observe that the claim holds trivially before the 
first iteration. In any iteration, if we assign job j to machine i in Step (ii) (b) then the cost 
of F increases by Cij and the current linear programming solution decreases by CijXij = Cij 
since Xij = 1. Hence, the claim holds. If we remove a constraint in Step (ii) (c) , then the 
cost of F remains the same, while the cost of the current linear program can only decrease. 
Hence, the claim holds in this case as well. Thus, finally when F is a feasible assignment, 
by induction, the cost of assignment given by F is at most the cost of the initial linear 
programming solution. 

Finally, we show that machine i is used at most 2Tj units for each i. Fix any 
machine i. We first argue the following claim. If i 6 M' , then at any iteration we must 
have T[ + Ti(F) < TJ, where T[ is the residual time left on the machine at this iteration 
and Ti(F) is the time used by jobs assigned to machine i in F. The proof of the claim 
follows by a simple inductive argument as in the above inductive argument for costs. Now 
consider when the machine i is removed from M' . There are two possibilities. If there 
is only one job j in machine i, then the total processing time at machine i is at most 
Ti + pij < 2Tj, where the inequality holds because of the pruning step (where we deleted 
edges assigning a job to machine i if its processing time exceeded Tj). If there are two 
jobs ji and j'2 in machine i then let x denote the linear programming solution when the 
constraint for machine i is removed. The total processing time at machine i at most 

T l (F)+p in +p in 

< Ti Xij 1 Pij 1 Xij 2 Pij 2 + Piji Viji 

< Ti + (1 — Xij 1 )pij 1 + (1 — Xij 2 )pij 2 

< + (2 ~ %ij 2 )Ti 

< 2Ti, 

because Pij 1 ,Pij 2 < Ti again by the pruning step and Xij 1 + x,- L j 2 > 1 by Step (ii) (c) . This 
completes the proof of Theorem 3.2.1. ■ 



3.3 Maximum Budgeted Allocation 

In this section we consider the maximum budgeted allocation problem, which is similar to 
the generalized assignment problem but is a maximization problem instead of a minimiza- 
tion problem. There are a set Q of indivisible items and a set A of agents. Each agent 
i G A is willing to pay a maximum of bij dollars for item j G Q, but has a maximum budget 
Bi on total spending. The maximum budgeted allocation problem is to allocate items to 
agents to maximize revenue. The main result of this section is the following theorem by 
Chakrabarty and Goel [27]. 

Theorem 3.3.1 There is a 4/ '3- approximation algorithm for the maximum budgeted allo- 
cation problem. 
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3.3.1 Linear Programming Relaxation 



This problem can be formulated as an integer linear programming, in which there is a 
variable x^ for agent i and item j to indicate whether item j is assigned to agent i. 

maximize min(.Bj , bijXij ) 

i&A jeQ 

subject to ~^2 x ij — 1 Vj G Q 

Xij G {0,1} Vi€A,j€Q 

The constraints require that each item is allocated to at most one agent. The objec- 
tive function can be rewritten as a linear function by adding auxiliary constraints. Thus 
we obtain a linear programming relaxation by relaxing the integrality constraints. Ob- 
serve that by appropriate scaling there is always an optimal fractional solution in which 
YljeQ bijXij < Bi for all i G A. Henceforth we consider the following equivalent linear 
programming relaxation, denoted by LP m b a , which is similar to that of the generalized 
assignment problem. Throughout this section we assume without loss of generality that 
bij < Bi for each j G Q (for otherwise, we can reset bij to Bi) and maintain this invariant 
in all iterations of the iterative algorithm. 



maximize ^ ^ bij x^ 

subject to hj x^ < Bi Vi£A 

j&Q 

Y^Xij < 1 V j G Q 

Xij > Vz G A, j G Q 



3.3.2 Characterization of Extreme Point Solutions 

Given a fractional solution x to LP m b a , we construct a bipartite graph G x = (A U Q,E) 
with ij G E if and only if Xij > 0. The following lemma is a direct application of the Rank 
Lemma 2.1.4. 

Lemma 3.3.2 Given any extreme point solution x to LP m \, a with x-ij > for each i G 
A,j G Q there exist A' C A, Q' C Q such that 

W T^jeQ bij x ij = Bi for each i G A' and X^ieA Xi 3 = ^ f or eac h 3 £ Q' ■ 
(ii) The corresponding row vectors in A' and Q' are linearly independent. 
(hi) \E\ = \A'\ + \Q'\. 
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Similar to the bipartite matching problem and the generalized assignment problem, 
the above lemma implies that each connected component of G x has at most one cycle. 
With an additional cycle elimination argument that shifts the values of the solution along 
such a cycle without decreasing the objective value, one can prove that G x is a forest (see 
exercises). We call an item a leaf item if it is a leaf in G x . We call an agent i a leaf 
agent if all but one neighbor of i are leaf items (i itself may not be a leaf). Since x is 
an extreme point solution and G x is a forest, there is at most one non-tight constraint in 
each component. For an agent that is a leaf of a tree, as its bid to the item is at most 
the agent's budget, such an agent must be non-tight in a tree with at least two agents (if 
such an agent is tight then the component has only this agent and this item) . In particular 
there cannot be two agents that are leaves of a tree. Hence, every tree in the forest has at 
least one leaf item and one leaf agent, and if a tree has at least two agents then there is at 
least one tight leaf agent. The following lemma summarizes the properties of the extreme 
point solutions that we will use. 

Lemma 3.3.3 [27] Given any extreme point solution x to LP m ^ a : 

(i) The graph G x is a forest; 

(ii) There is at least one leaf agent in each component of G x . 

(iii) There is at least one tight leaf agent in a component of G x with at least two agents. 



3.3.3 An Iterative 2- Approximation Algorithm 

We first present a simple iterative 2-approximation algorithm for the problem. In the 
following N(i) denotes the set of neighbors (items) of agent i in G x . 



Iterative 2- Approximation Algorithm for Maximum Budgeted Allocation 

While Q + do 

(i) Find an optimal extreme point solution x to LP m b a . Remove all edges with 

Xij — 0. 

(ii) Pick a leaf agent i. Let L be the set of leaf items in N(i). Assign each item I G L 
to i and then remove L. Modify B\ := Bi — YIi^l 

(a) If i has a non-leaf item j, modify the bid of i on j to be b\, <— min(bjj, £?■). 



Fig. 3.3. The Maximum Budgeted Allocation 2-Approximation Algorithm 

We now prove that the above algorithm is a 2-approximation algorithm. Consider 
an arbitrary iteration. The increase of the integral solution is at least YlieL^H x n- Note 
that x restricted to the remaining edges is a feasible solution to the residual problem. Let 
- = min(bij , B'j) denote the new bid of agent i to item j. If V- = by, then the decrease 
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of the fractional solution is at most YlitL ^U X U- Otherwise, if b'^ = then the decrease 
of the fractional solution is at most 

^2 hixu + {hj - b'i^xij < ^2 b u x u + ( B i - B i) = 2 ^2 bilXiU 
leL leL leL 

where the first inequality follows because hij < Bi and b\j = B[ and x^ < 1. Hence, 
in either case, the increase of the integral solution is at least half the decrease of the 
fractional solution in each iteration. It follows by an inductive argument that the final 
integral solution is at least half the initial fractional solution. 



3.3.4 An Iterative ^-Approximation Algorithm 

Here we present an improved iterative approximation algorithm for the problem. In 
Step (ii) (a) of the algorithm in Figure 3.3, the bid of agent i on item j is decreased 
by the amount collected from the leaf items, and this is the bottleneck of achieving a bet- 
ter approximation algorithm. The new idea is to keep the new bid slightly higher. The 
intuition is that there is some "surplus" from the amount collected from the leaf items, 
since they are collected with "factor one" while we only require an approximate solution. 
The improved algorithm is presented in Figure 3.4. 



Iterative ^-Approximation Algorithm for Maximum Budgeted Allocation 

While Q / do 

(i) Find an optimal extreme point solution x to LP m t, a . Remove all edges with 
Xij = 0. 

(ii) If there is a leaf agent i with all the items in N(i) being leaf items, then assign 
each item I G N(i) to i and remove N(i) and i. 

(iii) Pick a tight leaf agent i. Let L be the set of leaf items in N(i), and j be the 
unique non-leaf item in N(i). Assign each item in N(i) \ {j} to i and remove 
each item in N(i) \ {j}. Modify both 

4 

Bi,bij i T^bijXij. 



Fig. 3.4. The Maximum Budgeted Allocation 4/3- Approximation Algorithm 



3.3.5 Correctness and Performance Guarantee 

Lemma 3.3.3 guarantees that the algorithm will terminate successfully. Therefore, it re- 
mains to prove that the returned integral solution is at least | of the initial fractional 
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solution. As in the proof of the 2-approximation algorithm, we apply the following lemma 
inductively to complete the proof. 

Lemma 3.3.4 In any iteration the increase of the integral solution is at least | of the 
decrease of the fractional solution. 

Proof Consider any iteration and first suppose that Step (ii) is applied on a leaf agent 
i in this iteration. Let x denote the optimal fractional solution at the end of the itera- 
tion. Note that Bi is modified at most once throughout the algorithm. Suppose Bi has 
not been modified before this iteration. Then the increase of the integral solution is at 
least Yli£N(i)bil x il- Since the current solution x restricted to the remaining edges is a 
feasible solution in the residual problem, the decrease of the fractional solution is at most 
YlleNU) buXiu and thus the lemma holds. 

Now suppose that Bi has been modified before this iteration. In this case let j be 
the unique neighbor of i. Let y denote the fractional solution when Bi was modified. Let 
Bi denote the original budget, bij denote the original bid and b\j denote the current bid. 
The decrease in the fractional solution in the current step is at most its current bid 

t 4 

= -^bijDij. 

However, from the iteration when its budget was modified, the unspent budget left on 
agent i is at least bijt/ij and hence the increase of the integral solution is at least 

3 „ 
bijVij > - -bij. 

Therefore the lemma holds if Step (ii) is applied. 

Now, finally assume that Step (iii) (b) was applied on a tight leaf agent i. Let j be 
the unique non-leaf item in N(i). Then increase in the integral solution is at least 

^ ^ biiXn = Bi — bijXij. 

l€N(i)\{j} 

The decrease in the fractional solution is at most 

^ ] biixn + bijXij bijXij = Bi — bijXij = Bi — ~bijXij. 
ieN(i)\{j} 

Hence, the increase in the integral solution is at least 3/4 of the decrease of the fractional 
solution as verified below: 

3 

A(Integral) — -A(Fr actional Solution) 

3 4 2 

= Bi — bijXij — ~ ~^bijXij) 

- - h- x (i - x- ) > — - h -ii- > o 

— ^ VtjX t: j{± XjjJ ^ ^ u, 
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where the second last inequality uses the fact that x(l — x) < 1/4 for any < x < 1. This 
completes the proof of the lemma, and thus the proof of Theorem 3.3.1. □ 

3.4 Vertex Cover in Bipartite Graphs 

In this section we study the vertex cover problem in bipartite graphs. Given a graph 
G = (V, E) with cost function c : V — > M+ the vertex cover problem asks for a set of 
vertices V' such that e D V ^ for each e 6 E and c(V') = YlveV Cv ' s minimized. In 
this section we restrict our attention to bipartite graphs. 

3.4-1 Linear Programming Relaxation 

We first give the following natural linear programming relaxation LPb vc (G) for the vertex 
cover problem in bipartite graphs. We have a variable x v for each vertex v denoting its 
inclusion in the solution. 



minimize 2 c v x v 

subject to x u + x v > 1 Ve = {u, v } G E 

x v > o yv G V 

As in the previous two sections, the linear program LPb vc (G) is compact, i.e., the 
number of constraints and variables is polynomially bounded in the size of the problem. 
Hence, the linear program can be solved optimally in polynomial time by Theorem 2.1.6. 

We prove the following theorem by an iterative algorithm. 

Theorem 3.4.1 Given any cost function c there exists an integral vertex cover U such that 
c(U) < c ■ x where x is an optimal solution to LPb vc (G). 

As before, our proof of Theorem 3.4.1 as a corollary implies the following theorem. 

Theorem 3.4.2 The linear programming formulation LPb vc (G) is integral. 

3-4-2 Characterization of Extreme Point Solutions 

Before we prove Theorem 3.4.1 we give a characterization of extreme points of LP\y VC . For a 
set W C V, let x(W) denote the vector in M} v \\ the vector has an 1 corresponding to each 
vertex v £ W, and otherwise. This vector is called the characteristic vector of W, and 
is denoted by x(W)- ^ n the following lemma, which follows by a direct application of the 
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Rank Lemma 2.1.4, we characterize an extreme point solution by a set of tight independent 
constraints. 

Lemma 3.4.3 Given any extreme point x to LPb vc (G) with x v > for each v G V there 
exists F C E such that 

(i) x u + x v = 1 for each e = {u, v } G F . 

(ii) T/ie vectors in {x({u, v}) : G F} are linearly independent. 
(hi) \V\ = \F\. 

3.4-3 Iterative Algorithm 

We now give the algorithm which constructs an integral vertex cover of cost at most the 
optimal solution to LPb vc (G) proving Theorem 3.4.1. The algorithm is a simple iterative 
procedure and shown in Figure 3.5. 



Iterative Bipartite Vertex Cover Algorithm 




(i) Initialization U <— 0. 




(ii) While V(G) / do 




(a) Find an optimal extreme point solution x to LPb vc (G) 


and remove every 


vertex v with x v = and dE{v) = from G. 




(b) If there is a vertex v G V with x v = 1 then update U 


UU{v}, V(G) <- 


V(G) \ {u } and E(G) <- E(G) \ S(v). 




(iii) Return U. 





Fig. 3.5. Bipartite Vertex Cover Algorithm 



5.4-4 Correctness and Optimality 

Following the approach we used for bipartite matching, we prove the correctness of the 
algorithm in two steps. First, we show that the algorithm returns a vertex cover of optimal 
cost if the algorithm always finds a vertex v with x v = in Step (ii) (a) or a vertex v with 
x v = 1 in Step (ii) (b) . In the second part, we show that the algorithm will always find 
such an vertex completing the proof. 

Claim 3.4.4 If the algorithm, in every iteration, finds a vertex v with x v = and dE{v) = 
in Step (ii) (a) or a vertex v with x v = 1 in Step (ii)(b), then it returns a vertex cover U of 
cost at most the optimal solution to LPb vc (G). 

The proof of this claim is identical to the proof of Claim 3.1.4. We leave the details 
to the reader. We now complete the proof of Theorem 3.4.1 by showing that we can always 
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find a vertex v with x v = and <1e(v) = 0, or a vertex with x v = 1. The proof of the 
following lemma crucially uses the characterization of extreme point solutions given in 
Lemma 3.4.3. 

Lemma 3.4.5 Given any extreme point solution x to LP\, VC {G) there must exist a vertex 
v with x v = and (Ie(v) =0, or a vertex with x v = 1. 

Proof Suppose for sake of contradiction that x v < 1 and x v = implies that oIe(v) > 1 
for each vertex v £ V. This implies that there is no vertex v with = 0. This follows 
from the fact that any neighbor u of v must have x u = 1 since x a + x„ > 1. Lemma 3.4.3 
implies that there exists FOE such that constraints corresponding to F are tight and 
\F\ = \V\. 

We claim that F is acyclic. Suppose for sake of contradiction C C F is a cycle. 
Since G is bipartite C is an even cycle. Then C is the disjoint union of two matchings 
Mi and M2. But then the sum of the constraints corresponding to edges in Mi equals the 
sum of the constraints corresponding to edges in M2, contradicting the independence of 
constraints for edges in F. 

Since F is acyclic, we must have |F| < \V\ — 1, a contradiction. □ 

Thus, by Lemma 3.4.5, the algorithm in Figure 3.5 returns a vertex cover that costs 
at most the optimal solution to the linear program LP^ C (G), proving Theorem 3.4.1. 

3.5 Vertex Cover and Matching: Duality 

In this section we prove the following min-max theorem between the size of minimum 
vertex covers and maximum matchings in bipartite graphs using the integrality proofs we 
have seen in previous sections. 

Theorem 3.5.1 Given an unweighted bipartite graph G = (V,E) we have 
max{|M| : M is a matching} = min{|£7| : U is a vertex cover} 

Proof Let M* be the maximum weight matching returned by the Iterative Algorithm in 
Figure 3.1 when the weight function w e = 1 for all e G E. Also, let U* denote the minimum 
cost vertex cover returned by the iterative algorithm in Figure 3.5 when the cost function 
c v = 1 for each v G V. From Theorem 3.1.1 and Theorem 3.4.1 we have that M* is an 
optimal solution to LPb m (G) and U* is an optimal solution to LPb vc (G). But LPb m (G) and 
LPb vc (G) are duals of each other when w and c are uniformly one (see Section 2.1.4). By 
the strong duality theorem (Theorem 2.1.9) both the linear programs must have optimal 
solutions of equal value: 

|M*| = max{|M| : M is a matching} = min{|{7| : U is a vertex cover} = \U*\. 

□ 
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A weighted generalization of the above theorem is true. It can be similarly shown us- 
ing duality and the fact that the corresponding linear programming problems have integral 
optima. For more details, see Chapter 18 of Schrijver's book [150]. 

Notes 

The bipartite perfect matching problem (also known as the Assignment problem) is one 
of the oldest problems in Combinatorial Optimization (see e.g. [148]). The first proof of 
polynomial solvability of maximum weight bipartite matching via the Hungarian method 
was given by Egervary [51], sharpened by Kuhn [109] and shown efficient by Munkres [124]. 
The unweighted case was addressed earlier by Frobenius [60]. Birkhoff [19] was the first 
to show that the extreme points of the bipartite matching polyhedron (defined by the 
so-called "doubly stochastic" constraint matrix) are integral. 

The generalized assignment problem was first studied by Shmoys and Tardos [151] 
following up on a bi-criteria approximation algorithms for the problem due to Lin and 
Vitter [113]. Trick [157] studied the version minimizing the weighted sum of cost and 
makespan of the underlying scheduling problem, while Lenstra, Shmoys and Tardos [112] 
study the makespan problem and give the 2-approximation that was generalized in the 
subsequent work of Shmoys and Tardos [151]. Extensions have been considered by Saha 
and Srinivasan [145] and Zhang et al [166]; see exercises. 

The maximum budgeted allocation problem is NP-hard. The first approximation al- 
gorithm is a (1 + \/5) /2-approximation algorithm given by Garg, Kumar and Pandit [70]. 
The approximation ratio was subsequently improved to e/(e — 1) by Andelman and Man- 
sour [3], and 3/2 by Azar et al. [7]. The result presented in this chapter is by Chakrabarty 
and Goel [27]. The same result is also obtained independently by Srinivasan [155] by a 
dependent rounding algorithm. 

The bipartite minimum vertex cover problem is the linear programming dual to the 
maximum matching problem. The min-max theorem relating them is due to Konig [105]. 
The Hungarian method and LP duality can be used to show the integrality of the weighted 
vertex cover problem. An alternate approach is to use the total unimodularity (see e.g., 
the book by Nemhauser and Wolsey [130]) of the constraint matrix which is the edge-node 
incidence matrix of the bipartite graph, which is also an example of a Network Matrix that 
we will study in Chapter 8. 

Exercises 

3.1 Given a bipartite graph G = (V, E) and a cost function c : E — >■ 1Z, a perfect 
matching is a subgraph with degree exactly one at any node. 

(a) Write a linear programming formulation for the minimum cost perfect match- 
ing problem. 
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(b) Give an iterative proof of integrality of the linear programming formulation 
for the minimum cost perfect matching problem. (Hint: Adapt the proof for 
maximum weight matchings.) 

3.2 Show that LPb m (G) and LPb vc (G) are not integral when G is not bipartite. 

3.3 Generalize the methods given in this chapter for maximum weight matchings and 
minimum weight vertex covers in bipartite graphs to the case when the right hand 
side of the LP constraints are positive integers (rather than all l's as we consid- 
ered). Note that the two resulting integrality proofs, along with strong duality, will 
imply the general min-max relation between maximum- weight matchings with arbi- 
trary vertex degree bounds and minimum-cost vertex cover with arbitrary coverage 
requirements at the edges. 

3.4 Given a set of intervals [cij,&j] for each 1 < i < n and a weight function w on 
intervals, the maximum weight k-interval packing problem asks for a subset J of 
intervals of maximum weight such that there are at most k intervals in J at any 
point on the line. 

(a) Formulate a linear program for the maximum weight /c-interval packing prob- 
lem. 

(b) Show that only n — 1 point constraints need to be imposed apart from the 
bound constraints. 

(c) Show that the linear program is integral. 

3.5 Can you solve the maximum weight /c-interval packing problem for intervals on a 
tree rather than a path (as was the case in the previous problem) using the same 
approach? If you can, give an integral description for the problem. If not, argue 
why natural formulations are not integral. 

Consider the case when all the intervals in a tree are monotone, i.e., they 
go from a point in a rooted version of the tree to its ancestor (thus there are no 
intervals whose endpoints are two incomparable points in the tree). Can you give 
an integral LP formulation in this case using the approach of the previous problem? 

3.6 (Zhang et al [166], Saha and Srinivasan [145]) Consider the generalized assignment 
problem where for each machine i we are also given an upper bound U{ on the 
number of jobs that the machine i can service. Modify the iterative rounding 
algorithm to obtain the same guarantee as in Theorem 3.2.1 and also that the 
number of jobs on machine i is at most C/j. 

3.7 Prove Lemma 3.3.3 starting from Lemma 3.3.2. 

3.8 (Chakrabarty and Goel [27]) Construct an example for LP m t, a of the maximum 
budgeted allocation problem with integrality gap |. 
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4 

Spanning Trees 



t 

In this chapter we will study the spanning tree problem in undirected graphs. First, 
we will study an exact linear programming formulation and show its integrality using the 
iterative method. To do this, we will introduce the uncrossing method, which is a very 
powerful technique in combinatorial optimization. The uncrossing method will play a 
crucial role in the proof and will occur at numerous places in later chapters. We will show 
two different iterative algorithms for the spanning tree problem, each using a different 
choice of 1-elements to pick in the solution. For the second iterative algorithm, we show 
three different correctness proofs for the existence of an 1-element in an extreme point 
solution: a global counting argument, a local integral token counting argument and a local 
fractional token counting argument. These token counting arguments will be used in many 
proofs in later chapters. 

We then address the degree-bounded minimum-cost spanning tree problem. We show 
how the methods developed for the exact characterization of the spanning tree polyhedron 
are useful in designing approximation algorithms for this NP-hard problem. We give two 
additive approximation algorithm: the first follows the first approach for spanning trees 
and naturally generalizes to give a simple proof of the additive two approximation result 
of Goemans [72]; the second follows the second approach for spanning trees and uses the 
local fractional token counting argument to provide a very simple proof of the additive one 
approximation result of Singh and Lau [154]. 



4.1 Minimum Spanning Trees 

In an instance of the minimum spanning tree (MST) problem we are given an undirected 
graph G = (V, E), edge costs given as c : E — > R, and the task is to find a spanning tree 
of minimum total edge cost. 

t © Copyright 2011 by Lap Chi Lau, R. Ravi and Mohit Singh. Published 2011 by Cambridge 
University Press. 
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4-1.1 Linear Programming Relaxation 



In this section two formulations of the minimum spanning tree problem are discussed (some 
more are discussed in the Exercises). A spanning tree is a minimal 1-edge-connected 
subgraph. Thus, a natural formulation (sometimes called the undirected LP or the cut 
LP), is to require that every pair of vertices has a path connecting them. Or equivalently, 
we can require that there is at least one edge crossing each proper subset of vertices. 



minimize c e x e 

subject to x(5(S)) > 1 V S C V 

x e > Vee£ 



There is a variable x e corresponding to each edge e, to indicate that whether e is 
included in the spanning tree, and c e denotes the cost of e. For a set F of edges, the 
shorthand x(F) is used to denote ]C e e-F x e- Llecall that 5(S) is the set of edges with 
exactly one endpoint in S. 

There are exponentially many constraints in the undirected LP. However, from The- 
orem 2.1.8 in Chapter 1, the undirected LP can still be solved in polynomial time if a 
polynomial time separation oracle can be constructed. Constructing such a polynomial- 
time oracle is equivalent to finding a cut of total capacity less than one in the graph with 
capacities x, and can be accomplished using a global minimum-cut algorithm. (In detail, 
given a solution to the above linear program, one needs to determine whether it is a fea- 
sible solution, and if not, provides a violating inequality. A polynomial time separation 
oracle for the undirected LP is easy to construct. Given a fractional solution, check first if 
every variable is nonnegative. Then, for every pair of vertices, check if the maximum flow 
between them is at least 1. If this condition is satisfied for every pair, then clearly the given 
fractional solution is a feasible solution. On the other hand, if for some pair this condition 
is not satisfied, by the max-flow min-cut theorem, there is a set S with x(5(S)) < 1, and 
such a set can be found in polynomial time.) Unfortunately, the undirected LP is not an 
exact formulation of the minimum spanning tree problem as shown in Figure 4.1. 

Another formulation is the subtour elimination LP which is related to the study of 
the traveling salesman problem (TSP). For S C V, define E(S) to be the set of edges 
with both endpoints in S. For a spanning tree, there are at most |»S| — 1 edges in E(S), 
where denotes the number of vertices in S. Insisting on this for every set by using the 
constraint (4.2) eliminates all the potential subtours that can be formed in the LP solution: 
this is how the formulation gets its name. 
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Fig. 4.1. Consider a cycle of n vertices, where every edge has the same cost, say 1. Any spanning 
tree requires at least n — 1 edges, and thus has cost at least n — 1. However, by setting x e = 1/2 
for each edge, it can be easily checked that this is a feasible solution to the undirected LP, and has 
total cost only n/2. 



minimize ^^c e x e (4-1) 

subject to x(E(S)) < \S\ - 1 (4.2) 

x(E(V)) = \V\-1 (4.3) 
x e > V e £ E (4.4) 

In the next section we will give an iterative algorithm which will prove that the 
subtour LP is integral. 

Theorem 4.1.1 Every extreme point solution to the subtour LP is integral and corresponds 
to the characteristic vector of a spanning tree. 

Before we give the iterative algorithm and the proof of Theorem 4.1.1, we show 
that one can optimize over the subtour LP in polynomial time. We show this by giving 
a polynomial time separation oracle for the constraints in subtour LP. Polynomial time 
solvability now follows from Theorem 2.1.8. 

Theorem 4.1.2 There is a polynomial time separation oracle for the subtour LP. 

Proof Given a fractional solution x the separation oracle needs to find a set S C V such 
that x(E(S)) > \S\ — 1 if such a set exists. It is easy to check the equality x(E(V)) = 
\V\ — 1. Thus, checking the inequality for each subset S C V is equivalent to checking 
mins c y{|S| - 1 - x(E(S))} < 0. Using x(E(V)) = \V\ - 1 we obtain that it is enough 
to check minsdSI - 1 + x{E{V)) - x{E(S))} < \V\ - 1}. We show that solving 2\V\ - 2 
min-cut problems suffice to check the above. 

Fix a root vertex r £ V. For each k £ V \ {r}, we construct two minimum cut 
instances, one which checks the inequality for all subsets S containing r but not k and 
the other checks the inequality for all subsets S containing k but not r. We outline the 
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construction for the first one, the second construction follows by changing the roles of r 
and k. 




Fig. 4.2. The above figure shows the construction for separation of spanning tree constraints. Solid 
edges are part of the cut and dashed edges are not part of the cut. Thick edges into k have cost 
1 and thick edges from r to any node j cost J2 e eS(j) if - Thin arcs between nodes i and j form a 
pair of anti-parallel edges of cost X{ '^ } . 

We construct a directed graph G with vertex set V and arcs and for each 
edge {i,j} in G. We let the weight of edges (i, j) and (j, i) to be . We also place arcs 
from each vertex v G V \ {r, k} to k of weight 1 and arcs from r to each vertex v G V \ {r} 
of weight YleeS(v) it = ■ Consider any cut (S, V \ S) which separates r from k. 

Edges of weight one contribute exactly \S\ — 1. The edges between i and j of weight ^j- 
contribute exactly x( - s ^' ) . The edges from r to rest of the vertices contribute YlvgS X ^ S ^ ■ 
Thus the total weight of the cut is exactly 

\ S \ - 1 + + e ^ = w - 1 + x ( E <y» - x ^ s »- 

Hence, checking whether the minimum cut separating r from k is strictly smaller than 
\V\ — 1 checks exactly whether there is a violating set S containing r but not containing 
k. □ 



4-1-2 Characterization of Extreme Point Solutions 

In this subsection, we analyze the extreme point solutions to the subtour LP. Recall that 
an extreme point solution is the unique solution defined by n linearly independent tight 
inequalities, where n is the number of variables in the linear program. There are expo- 
nentially many inequalities in the subtour LP, and an extreme point solution may satisfy 
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many inequalities as equalities. To analyze an extreme point solution, an important step 
is to find a "good" set of tight inequalities defining it. If there is an edge e with x e = 0, 
this edge can be removed from the graph without affecting the feasibility and the objective 
value. So henceforth assume every edge e has x e > 0. 



4-1-3 Uncrossing Technique 

The uncrossing technique is a powerful technique and we shall use it to find a good set 
of tight inequalities for an extreme point solution in the subtour LP. For a set F C E, 
let x{F) denote the characteristic vector in M} E \ that has an 1 corresponding to each edge 
e G F, and otherwise. The following proposition follows from the supermodularity of 
see also Proposition 2.3.6. 

Proposition 4.1.3 For X,Y C V, 

X(E(X)) + x(E(Y)) < X (E(X U Y)) + X (E(X n Y)), 
and equality holds if and only if E(X \ Y,Y \ X) = 0. 

Proof Observe that 

X(E(X)) + x(E(Y)) = X (E(X U Y)) + X (E(X n Y)) - X (E(X \Y,Y \ X)) 
and proof follows immediately. See Figure 4.3. □ 




Fig. 4.3. In this example the solid edges are counted exactly once in both the LHS xC^PO) + 
X(E(Y)), and the RHS x( E ( x u Y )) + x( E ( x n Y)), and the bold edge is counted exactly twice 
on both sides. The dashed edge, however is counted in the RHS but not in the LHS. 

Given an extreme point solution x to the subtour LP, let T = {S C V \ x(E(S)) = 
\S\ — 1} be the family of tight inequalities for x. The following lemma shows that this 
family is closed under intersection and union. 
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Lemma 4.1.4 If S,T £ J 7 and SdT / 0, then both SdT and SUT are in T . Furthermore, 
X(E(S)) + X (E(T)) = X(E(S n T)) + X (E(S U T)). 

Proof Observe that 

|5|-1 + |T|-1 = x(E(S))+x(E(T)) 

< x(E(S fl T)) + x(E(S U T))) 

< |5nr| - 1 + |5ur| - 1 

= |5|-1 + |T|-1. 

The first equality follows from the fact that 5, T G T . The second inequality follows 
from Proposition 4.1.3 (or from Proposition 2.3.6). The third inequality follows from 
the constraints for S (1 T and S U T in the subtour LP. The last equality is because 
1 5 1 + \T\ = \S fl T\ + \S U T\ for any two sets S, T. Thus equality must hold everywhere 
and we have x(E(S n T)) + x(E(S UT)) = \S n T\ - 1 + \S U T| - 1. Hence, we must 
have equality for constraints for S C\T and S UT, i.e., x(E(S D T)) = |5 fl T| — 1 and 
x(E(S U T)) = |5 U T| — 1, which implies that S P\T and S UT are also in J 7 . Moreover, 
equality holds for Proposition 4.1.3 and thus x(^(-5 \T,T \ S)) = and x(-E(S')) + 
X (E(T)) = x(E(SnT))+x(E(SUT)). □ 

Denote by span{F) the vector space generated by the set of vectors { X (E(S)) \ S € 
J 7 }. Call two sets X, Y intersecting if X fl Y, X — Y and Y — X are nonempty. A 
family of sets is laminar if no two sets are intersecting. The following lemma says that an 
extreme point solution is characterized by tight inequalities whose corresponding sets form 
a laminar family. This is a crucial structure theorem on the extreme point solutions for 
the subtour LP. 

Lemma 4.1.5 If £ is a maximal laminar subfamily of T , then span(C) = span(J-). 

Proof Suppose, by way of contradiction, that £ is a maximal laminar subfamily of T but 
span(C) C span{F). For any S £ C, define inter sect(S, C) to be the number of sets in 
C which intersect S, i.e. inter sect(S, C) = \{T £ C \ S and T are intersecting}). Since 
span(C) C span(F), there exists a set S with X (E(S)) ^ span(C). Choose such a set S 
with minimum intersect(S, C). Clearly, inter sect(S, C) > 1; otherwise C U {S} is also a 
laminar subfamily, contradicting the maximality of C Let T be a set in C which intersects 
S. Since S,T <= F, by Lemma 4.1.4, both SflT and SUT are in F. Also, both intersected 
T,C) and inter sect(S U T, C) are smaller than inter sect(S, £), which will be proved next 
in Proposition 4.1.6. Hence, by the minimality of inter sect(S, £), both S (~)T and S U T 
are in span(£). By Lemma 4.1.4, x(E(S)) +x(E(T)) = x(E(S DT)) + X (E(SUT)). Since 
X (E(S P\T)) and x{E{S U T)) are in span(£) and T £ C, the above equation implies that 
x{E(S)) G span(C), a contradiction. It remains to prove Proposition 4.1.6. 

Proposition 4.1.6 Zei S be a set that intersects T € C. Then inter sect(S flT, £) and 
inter sect(S U T, £) are smaller than inter sect(S, C). 
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Proof Since C is a laminar family, for a set R £ C with R 7^ T, R does not intersect T 
(either R (Z T, Tci?orTni? = 0). So, whenever i? intersects S* D T or 5 U T, i? also 
intersects S. Also, T intersects S but not 5 f~1 T or 5 U T. Therefore, intersect(S n T, £) 
and inter sect{S U T, £) are smaller than intersect(S, £) □ 

This completes the proof of Lemma 4.1.5. □ 

The following proposition about the size of a laminar family will be used to bound 
the number of variables in an extreme point solution. 

Proposition 4.1.7 A laminar family C over the ground set V without singletons (subsets 
with only one element) has at most \V\ — 1 distinct members. 

Proof The proof is by induction on the size of the ground set. If |V| = 2, clearly the claim 
follows. Let n = \ V\ and the claim be true for all laminar families over ground sets of size 
strictly smaller than n. Let 5 be a maximal set in the laminar family which is not equal 
to V. Each set in C, except for V, is either contained in S or does not intersect S. The 
number of sets in C contained in S (including S itself) is at most \S\ — 1 by the induction 
hypothesis. The sets in C not intersecting with S form a laminar family over the ground 
set V \ S and hence there are at most \V\ — \S\ — 1 such sets. Along with V, this gives a 
total of at most (\S\ - 1) + (|V| - \S\ - 1) + 1 = \V\ - 1 sets. □ 

The following corollary follows immediately from Proposition 4.1.7. 

Corollary 4.1.8 A laminar family C over the ground set V (potentially including singletons 
now) has at most 2|V| — 1 distinct members. 



4-1-4 Leaf-finding Iterative Algorithm 

In this subsection, an iterative procedure to find a minimum spanning tree from an optimal 
extreme point solution of the subtour LP is presented. The algorithm is shown in Figure 4.4. 

Iterative Leaf-finding MST Algorithm 

(i) Initialization F 0. 

(ii) While \ V(G)\ > 2 do 

(a) Find an optimal extreme point solution x to the subtour LP and remove 
every edge e with x e = from G. 

(b) Find a vertex v with at most one edge e = uv incident on it, then update 
F <- F U {e} and G <- G \ {v}. 

(iii) Return F. 

Fig. 4.4. Iterative Leaf-finding MST Algorithm 
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4-1-5 Correctness and Optimality of Leaf-Finding Algorithm 

First, we prove that the algorithm will terminate in the following lemma. 

Lemma 4.1.9 For any extreme point solution x to the subtour LP with x e > for every 
edge e, there exists a vertex v with d{v) = 1. 

Proof Suppose each vertex is of degree at least two. Then \E\ = \ YlveV d{ v ) > 1^1- On 
the other hand, since there is no edge e with x e = 0, every tight inequality is of the form 
x(E(S)) = \S\ — 1. By Lemma 4.1.5, there are \C\ linearly independent tight constraints of 
the form x(E(S)) = \S\ — 1, where C is a laminar family with no singleton sets. It follows 
that \E\ = \C\ by the Rank Lemma 2.1.4. By Proposition 4.1.7, \C\ < \V\ - 1 and hence 
\E\ < \V\ — 1, a contradiction. □ 

Next we show that the returned solution is a minimum spanning tree in the following 
theorem. 

Theorem 4.1.10 The Iterative MST Algorithm returns a minimum spanning tree in poly- 
nomial time. 

Proof This is proved by induction on the number of iterations of the algorithm. If the 
algorithm finds a vertex v of degree one (a leaf vertex) in Step (ii) (b) with an edge e = {u, v} 
incident at v, then we must have x e = 1 since x(5(v)) > 1 is a valid inequality of the LP 
(subtracting the constraint x{E(V — v)) < \V\ — 2 from the constraint x(E(V)) = \V\ — 1). 
In the iterative leaf-finding algorithm, e is added to the solution F (starting with an 
initial F = 0), and v is removed from the graph. Note that for any spanning tree T' of 
G' = G \ {v}, we can construct a spanning tree T = T' U {e} of G. Hence, the residual 
problem is to find a minimum spanning tree on G\d, and the same procedure is applied 
to solve the residual problem recursively. 

Since x e = 1, the restriction of x to E(G'), denoted by x res , is a feasible solution 
to the subtour LP for G' . Inductively, the algorithm will return a spanning tree F' of G' 
of cost at most the optimal value of the subtour LP for G', and hence c(F') < c • x res . 
Therefore, 

c(F) = c(F') + c e and c(F') < c ■ x res 

which imply that 

as x e = 1. Hence, the spanning tree returned by the algorithm is of cost no more than 
the cost of an optimal LP solution x, which is a lower bound on the cost of a minimum 
spanning tree. This shows that the algorithm returns a minimum spanning tree of the 
graph. □ 
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Remark 4.1.11 If x is an optimal extreme point solution to the subtour LP for G, then 
the residual LP solution x res , x restricted to G' = G\v , remains an optimal extreme point 
solution to the subtour LP for G' . Hence, in the Iterative MST Algorithm we only need to 
solve the original linear program once and none of the residual linear programs. 

Theorem 4.1.10 also shows that the subtour LP is an exact formulation of the mini- 
mum spanning tree problem showing the proof of Theorem 4.1.1. 

Alternately, note that the proof of Lemma 4.1.9 already showed that \E\ = n — 1 
and since x(E) = n — 1 and x(e) < 1 for all edges e 6 E (by considering the constraint 
x(E(S)) = 151 — 1 for the size-two set S defined by the endpoints of the edge), we must have 
x e = 1 for all edges e 6 E proving integrality. Thus we have that directly either x e = 
or x e = 1 for all edges e rather than for a single edge. This gives a direct (non-iterative) 
proof of integrality of the subtour LP. 

4.2 Iterative 1-edge-finding Algorithm 

In this section, we give another iterative procedure to find a minimum spanning tree from 
an optimal extreme point solution to the subtour LP is presented. The algorithm is shown 
in Figure 4.5. A key difference is that to create the residual problem, the chosen edge e is 
contracted from G to identify its endpoints to result in the graph G/e. 



Iterative 1-edge-finding MST Algorithm 




(i) Initialization F 0. 




(ii) While \ V(G)\ > 2 do 




(a) Find an optimal extreme point solution x to the subtour LP and 


remove 


every edge e with x e = from G. 




(b) Find an edge e = {u, v} with x e = 1 and update F F U {e}, G 


<- G/e. 


(iii) Return F. 





Fig. 4.5. Iterative 1-edge-finding MST Algorithm 



4-2.1 Correctness and Optimality of 1 -Edge- Finding Algorithm 

Following the discussion in Subsection 4.1.5 it is enough to show that the algorithm will 
terminate. An argument similar to one in proof of Theorem 4.1.10 will show that the 
output of the algorithm is a minimum spanning tree. 

Lemma 4.2.1 For any extreme point solution x to the subtour LP with x e > for each 
edge e there exists an edge f with Xf = 1. 
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Proof We give three alternate proofs of this lemma mainly to illustrate the three types of 
counting arguments that can be used to accomplish such proofs. 

Proof 1 (Global Counting Argument). This is the proof style of Lemma 4.1.5 which 
shows by a global degree counting argument over all (non-leaf) nodes that \E\ > \ V\, which 
contradicts the upper bound of \C\ of |V| — 1. 

Proof 2 (Local Token Counting Argument). By Lemma 4.1.5, there are |£| linearly 
independent tight constraints of the form x{E(S)) = \S\ — 1, and so \E\ = \C\. We now 
show a contradiction to this through a local token counting argument. 

We assign one token for each edge e in the support E, for a total of \E\ tokens. We 
will redistribute the tokens so that each set in C will receive one token and there are some 
extra tokens left. This implies that \E\ > |L|, giving us the contradiction. Actually, for 
the contradiction, we can collect two tokens for each set S G C. Since \C\ > 1, this will 
give the desired extra token and hence the contradiction. 

To redistribute the tokens, each edge gives its token to the smallest set containing 
both of its endpoints. Let S be any set in C with children R\, . . . (k could be zero). 
We have 

x(E(S)) = \S\-l 

and for each i, 

x(E(R t )) = \Ri\ - 1. 

Subtracting, we obtain 

k k 

x{E{S)) - x(E(IU)) = \S\ - +k-l. 

i=l i=l 

Let A = E(S) \ (UiE(Ri)). Observe that S obtains exactly one token for each edge in A. 
If A = 0, then x(E(S)) = x(E(Ri)) which contradicts the linear independence of these 
constraints in C. Moreover, \A\ ^ 1 as x(A) is an integer but no single edge in it has an 
integral value. Hence, \A\ > 2 and thus S receives at least two tokens. 

Proof 3 (Local Fractional Token Counting Argument). This is a slight modification 
of the previous argument but generalizes nicely to the degree-bounded case. As before, we 
assign one token for each edge e in the support E, for a total of \E\ tokens. For each edge 
e, however, we only redistribute x e fractional token to the smallest set containing both the 
endpoints. Now, we show that each set in C can collect at least one token, and demonstrate 
some extra leftover fractional edge tokens as before giving us the contradiction. 

Let S be any set in C with children R±, . . . , for some k > 0. Following the previous 
proof, we have that 

x(E(S)) -Y^x(E(Ri)) = \S\-J2 1^1 + k ~ 1 

i i 

=► x{A) = \S\-Y J \Ri\ + k-l 
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where A = E(S) \ (UiE(Ri)). Now S obtains exactly x e fractional token for each edge e 
in A. If A = 0, then x(E(S)) = YliX(E(Ri)) which contradicts the linear independence 
of these sets of constraints in C. Moreover, x(A) is an integer and hence it is at least one, 
giving S the one token it needs. 

Since every edge is not integral, we have the extra fractional token of value (1 — x e ) 
for every edge e as unused tokens giving the contradiction. 



We next turn to the study of the minimum bounded-degree spanning tree (MBDST) prob- 
lem. In an instance of the MBDST problem we are given a graph G = (V,E), edge cost 
given by c : E — > R, a degree upper bound B v for each v £ V and the task is to find a 
spanning tree of minimum cost which satisfies the degree bounds. We prove the following 
theorem originally due to Singh and Lau [154]. 

Theorem 4.3.1 There exists a polynomial time algorithm which given a feasible instance 
of the MBDST problem returns a spanning tree T such that cLt(v) < B v + 1 and cost of the 
tree T is at most the cost of any tree which satisfies the degree bounds. 

We prove Theorem 4.3.1 using the iterative relaxation technique. However, we first 
prove a weaker guarantee where the degree bound is violated by an additive factor of two, 
a result first obtained by Goemans [72] and illustrates a simple extension of the leaf-finding 
iterative MST algorithm. Later in Section 4.4, we present the proof of Theorem 4.3.1 that 
extends the 1-edge-fmding iterative MST algorithm. 



We use the following standard linear programming relaxation for the MBDST problem, 
which we denote by LP m bd s t(G, £>, W). In the following we assume that degree bounds are 
given for vertices only in a subset W C V . Let B denote the vector of all degree bounds 
B v , one for each vertex v £ W. 



□ 



4.3 Minimum Bounded-Degree Spanning Trees 



4-3.1 Linear Programming Relaxation 



minimize 




(4.5) 



subject to 



x(E(V)) = \V\-1 
x(E(S)) < \S\-l 
x(8(v)) < B v 



Vt) G W 



(4.6) 
(4.7) 
(4.8) 
(4.9) 



X e > 



Ve 6 E 
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Separation over the inequalities in the above linear program can be carried out in 
polynomial time and follows from Theorem 4.1.2. An alternative is to write a compact 
reformulation of the above linear program which has polynomially many variables and 
constraints (see the exercises). When the above LP relaxation is infeasible, the method 
signals infeasibility; otherwise, it proceeds to iteratively round and provide an approximate 
solution. Henceforth we assume that we are working with a feasible instance of the problem 
for which there is trivially a feasible LP solution. 



4-3.2 Characterization of Extreme Point Solutions 

We first give a characterization of an extreme point solution to LP m i,^ st (G, B, W). We 
remove all edges with x e = and focus only on the support of the extreme point solution 
and the tight constraints from (4.6)-(4.8). Let T = {S C V : x(E(S)) = \S\ - 1} be the set 
of tight constraints from (4.6)-(4.7). From an application of Rank Lemma 2.1.4 and the 
characterization of extreme point solutions to the spanning tree polyhedron (Lemma 4.1.5), 
we have the following characterization. 

Lemma 4.3.2 Let x be an extreme point solution to LP m bdst(G,B,W) with x e > for 
each edge e G E. Then there exists a set T C W and a laminar family C such that 

(i) x(5(v)) = B v for each v G T and x(E(S)) = 15*1 — 1 for each S G C. 

(ii) The vectors in {x(E(S)) : S G £} U {x(3( v )) '■ v £ T} are linearly independent. 
(hi) \C\ + \T\ = \E\. 

4-3.3 Leaf-finding Iterative Algorithm 

In this section, we give an iterative polynomial-time algorithm which returns a tree of 
optimal cost and violates the degree bound within an additive error of two. The algorithm 
is given in Figure 4.6 



4-3-4 Correctness and Performance Guarantee 

In the next lemma we prove by a simple counting argument that in each iteration we can 
proceed by applying either Step (ii) (b) or Step (ii) (c) ; this will ensure that the algorithm 
terminates. 

Lemma 4.3.3 Any extreme point solution x to LP m bd s t(G, B,W) with x e > for each 
edge e G E must satisfy one of the following. 

(a) There is a vertex v G V with d^(f) = 1. 

(b) There is a vertex v G W with dE{v) < 3. 
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Iterative MBDST Leaf-finding Algorithm 

(i) Initialization F <— 0. 

(ii) While \ V{G)\ > 2 do 

(a) Find an optimal extreme point solution x to LP m ^ st (G, £>, W) and remove 
every edge e with x e = from G. Let the support of x be 22. 

(b) If there exists a vertex v £ V with at most one edge e = uv incident on 
it, then update F <- F U {e}, G <- G \ {v}, W <- W \ {v}, and set 
B u ^— B u — 1. 

(c) (Relaxation) If there exists a vertex v £ W with cIe(v) < 3, then update 
W ^ W\{v}. 

(iii) Return F. 



Fig. 4.6. MBDST Leaf-finding Algorithm 

Proof We use a global counting argument. Suppose for sake of contradiction that both 
(a) and (b) are not satisfied. Then every vertex has at least 2 edges incident on it and 
every vertex in W has at least 4 edges incident on it. Therefore, \E\ = ^YlveV d( v ) — 
5(2(n- |W|) + 4|W|) = n+|W|, where n=\V\. 

On the other hand, by Lemma 4.3.2, there is a laminar family C and a set T C W 
of vertices such that \E\ = \C\ + \T\. As £ is a laminar family which contains subsets of 
size at least two, from Proposition 4.1.7 we have \C\ < n — 1. Hence, \E\ = \C\ + |T| < 
n - 1 + \T\ < n - 1 + \ W\, a contradiction. □ 

We now prove the performance guarantee of the algorithm. 

Theorem 4.3.4 The leaf-finding iterative algorithm in Figure 4-6 returns a tree T of 
optimal cost with cLt(v) < B v + 2 for each v £ V. 

Proof The proof that the cost of tree returned is at most the cost of the linear programming 
solution is identical to the proof of the optimality of the iterative spanning tree algorithm 
in Section 4.1 and we do not duplicate it here. 

We show that the degree of any vertex v is at most B v + 2. At any iteration, let F 
denote the set of edges selected and let B' v denote the current residual degree bound of v. 
We claim that while the degree constraint of v is present, oIf{v) + B' v = B v . The proof is 
by induction on the number of iterations of the algorithm. Clearly, F = and B' v = B v 
and the claim holds. At any iteration, whenever we include an edge e G 5(v) in F, we 
reduce B' v by one and hence the equality holds true. 

When the degree bound for the vertex v is removed then at most 3 edges are incident 
at v. In the worst case, we may select all three edges in the solution. Hence, 

d T (v) < B v - B' v + 3 < B v + 2 

where B' v > 1 is the degree bound of v when the degree constraint is removed. □ 
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4.4 An Additive One Approximation Algorithm 

In this section, we give a very simple iterative algorithm which returns a tree of optimal 
cost and violates the degree bound within an additive error of one. This algorithm involves 
no rounding at all (not even picking an 1-edge) - it removes degree constraints one by one, 
and eventually reduces the problem to a minimum spanning tree problem. This can be 
thought of as an analogue of the 1-edge-finding iterative MST algorithm presented earlier. 
The algorithm is given in Figure 4.7. 

Iterative Relaxation MBDST Algorithm 

(i) While W / do 

(a) Find an optimal extreme point solution x to LP m ^ st (G, B, W) and remove 
every edge e with x e = from G. Let the support of x be E. 

(b) (Relaxation) If there exists a vertex v G W with cIe(v) < B v + 1, then 
update W <- W \ {v}. 

(ii) Return E. 

Fig. 4.7. Additive One MBDST Algorithm 



4-4-1 Correctness and Performance Guarantee 

In the next lemma we prove that in each iteration, the algorithm can find some vertex for 
which the degree constraint can be removed. Observe that once all the degree constraints 
are removed we obtain the linear program for the minimum spanning tree problem which 
we showed in Section 4.1 to be integral. Hence, the algorithm returns a tree. Moreover, 
at each step we only relax the linear program. Hence, the cost of the final solution is at 
most the cost of the initial linear programming solution. Thus the tree returned by the 
algorithm has optimal cost. A simple inductive argument also shows that the degree bound 
is violated by at most an additive one. The degree bound is violated only when we remove 
the degree constraint and then cIe(v) < B v + 1. Thus, in the worst case, if we include all 
the edges incident at v in T, the degree bound of v is violated by at most an additive one. 

It remains to show that the iterative relaxation algorithm finds a degree constraint 
to remove at each step. From Lemma 4.3.2 we have that there exists a laminar family 
C C T and T C W such that \C\ + \T\ = \E\ and constraints for sets in C are linearly 
independent. Observe that if T = then only the spanning tree inequalities define the 
solution x. Hence, x must be integral. In the other case, we show that there must be a 
vertex in W whose degree constraint can be removed. 

Lemma 4.4.1 Let x be an extreme point solution to LP m ^ S f(G, B,W) with x e > 0. 
Let £, and T C W correspond to the tight set constraints and the tight degree constraints 
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defining x as given by Lemma 4-3.2. If T 7^ then there exists some vertex v G W with 
d E (v) <B V + 1. 

Proof We use the local fractional token argument as in the integrality proof of the 1-edge- 
flnding iterative MST algorithm we presented earlier. 

Suppose for the sake of contradiction, we have T 7^ and c2e(v) > B v + 2 for each 
v G W. We now show a contradiction by a fractional token argument. We give one token 
for each edge in E. We then redistribute the token such that each vertex in T and each set 
in C gets one token and we still have extra tokens left. This will contradict \E\ = \T\ + \C\. 
The token redistribution is as follows. Each edge e G E gives as before x e fractional token 
to the smallest set in C containing both endpoints of e, and (1 — x e )/2 fractional token to 
each of its endpoints for the degree constraints. 

We have already argued earlier that the x e assignment suffices to obtain one token 
for each member in the laminar family (see the third fractional token argument in the proof 
of Lemma 4.2.1). 

Thus it suffices to show that each vertex with a tight degree constraint gets one 
token. Let v G T be such a vertex. Then v receives (1 — x e )/2 token for each edge incident 
at v for a total token of value 

E l-Xe _ d E (v) ~ By > 
2 2 ~ ' 

e£8(v) 

where the first equality holds since ^ee<5(«) Xe = an( ^ the inequality holds since dE{v) > 
B v + 2 by Step (i)(b) of the algorithm. 

To finish the proof, we argue that there is some extra token left for contradiction. 
If V £ C then there exists an edge e which is not contained in any set of C and the x e 
token for that edge gives us the contradiction. Similarly, if there is a vertex v 6 W \ T 
then v also collects one token which it does not need and we get the desired contradiction. 
Moreover, if there is a vertex v G V \ T then each edge e incident at v must have x e = 1 
else the (1 — x e )/2 > token is extra. Note that e G span(C) for each e with x e = 1, since 
e is a tight set of size two. We have 

2 x (E(V)) = J2x(S(v)) = J2x(S(v))+ £ x (S(v)) = J2x(S(v))+ £ £ x ( e ). 
vev vgt vev-T »er v£V-T e eS(v) 

We have argued that V G C and e G span(C) for each edge e G for v G V — T. Since 
T / 0, this implies the linear independence of the tight constraints in T and those in £, 
giving us the contradiction. □ 



Notes 

Many variants of the greedy algorithm for finding minimum spanning trees have been 
obtained starting from Boruvka [21], Kruskal [108] and Prim [135] (Graham and Hell [75] 
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have a useful survey of the history). Edmonds [48] gave the integral linear programming 
relaxation for minimum spanning tree problem that we presented. 

There is a long line of work of successively improving the performance guarantees for 
the degree-bounded minimum-cost spanning tree problem. The algorithm with additive 
guarantee of one for the unweighted case was first given by Ftirer and Raghavachari [66]. 
The additive algorithm with violation 2 (with both upper and lower degree bounds) was 
presented by Goemans [72]. The algorithm with additive violation of 1 was first presented 
by Singh and Lau [154], also for the case with upper and lower bounds on the degree. 
The fractional token proof which we used for the additive one proof was first presented 
by Bansal et al. [9]. Chan et al [29] consider the degree bounded spanning tree problem 
with metric costs and give true approximation by applying splitting off techniques to the 
solution presented here (see exercises). 



Exercises 

4.1 (Partition LP formulation for spanning trees.) Consider the following par- 
tition LP for the minimum spanning tree problem. Let tt = {V±, . . . ,Vi} be a 
partition of the vertex set V, and let \tt\ = I denote the size of the partition. Define 
A(ir) to be the set of edges with endpoints in different sets in the partition tt. In 
any spanning tree, there are at least \n\ — 1 edges in A(tt) for a partition tt of V. 
Note that the undirected LP is a special case where only partitions of size two are 
considered. Show that the partition LP is equivalent to the subtour LP. 



minimize c e x e 

subject to x(A(ir)) > \ir\ — 1 for all partitions it of V 

x(E(V)) = \V\-1 

x e > V e G E 

4.2 Let r = {S C V : x(E(S)) = \S\ — 1} where x is an extreme point solution to the 
spanning tree polyhedron. Solving the linear program using the ellipsoid method 
enables us to get a set family JCr such that constraints for sets in T are linearly 
independent and span all the tight constraints in r. But T need not be laminar. 
Give a polynomial time algorithm that, given returns a laminar family C C r 
such that constraints for sets in C are independent and span all the tight constraints 
in r. 

4.3 (Spanning tree formulations.) Another interesting formulation for the mini- 
mum spanning tree problem is the bidirected LP, which models the problem as 
finding a spanning arborescence (i.e. a directed spanning tree). Given a directed 
graph D and a root vertex r, a spanning r- arborescence is a subgraph of D so that 
there is a directed path from r to every vertex in V — r. Given an undirected graph 
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G, we construct a directed graph D by having two opposite arcs (u, v) and (v, u) for 
each undirected edge uv in G, and the cost of (u, v) and (v, u) in D are the same as 
the cost of uv in G. Then, pick an arbitrary vertex r as the root vertex, and require 
that there is a directed path from r to every vertex in V — r. Or equivalently, by 
Menger's theorem, require that there is at least one arc entering every set which 
does not contain the root vertex. The bidirected LP formulation for the minimum 
spanning tree problem is shown as follows. 

minimize ^ 

aeA 

subject to x(5 in (S)) > 1 VSC^-r 
x a > V a £ A 

It is not clear a-priori whether the different spanning tree formulations are exact 
(i.e., give integer solutions at all extreme points) or not. (For example, consider 
the bidirected LP which looks almost the same as the undirected LP.) Also, it is 
not clear whether they can be solved in polynomial time. The following exercises 
shows that these three formulations are equivalent. 

(a) Suppose all edge weights are positive. Then show that the partition LP, the 
subtour LP, and the bidirected LP are equivalent. That is, any solution of 
one can be translated into a solution of the other with the same cost. 

(b) Using the above result, provide separation oracles for these formulations and 
show that there is a polynomial time separation oracle for the partition LP, 
the subtour LP, and the bidirected LP. 

4.4 Argue that in the iterative relaxation algorithm in Figure 4.7, one only needs to 
compute an optimal extreme point solution once initially, after that one can modify 
the current solution to obtain an extreme point solution for the next iteration in 
a simple way. (Hint: After we relax a degree constraint, we only need to find 
another extreme point solution with cost no more than the original solution, and 
the current solution is an "almost" extreme point solution for the next iteration.) 

4.5 (Ghodsi et al [77]) In an instance of the minimum bounded weighted-degree span- 
ning tree we are given a graph G = (V, E) and cost function c : E — >■ M + , a weight 
function w : E — > M + , a degree bound B v on each vertex v, and the task is to find 
a spanning tree T with minimum cost and YleeS T M w ( e ) ^ &v for all v £ V. Give 
a good bi-criteria approximation algorithm for the problem. 

4.6 Can you generalize the result in the previous problem to get good bi-criteria approx- 
imation algorithm for the minimum bounded weighted-degree Steiner tree problem? 
Why or why not? 

4.7 (Chan et al [29]) Consider the minimum bounded degree spanning tree problem 
when the cost function c : E — > Z satisfies triangle inequalities, i.e. c(uv) +c(vw) > 
c(uw) for all u,v,w £ V. Given a spanning tree T, denoted by driv) the degree 
of a vertex v in T and B v the degree bound for v. Using a minimum-cost flow 
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technique, Fekete et al [52] showed that the cost of a tree satisfying all degree 
bounds is at most the cost of T times 

2 - mm{ ^^2 :VGy '^ )>2} - 

Use this result and Theorem 4.3.1 to prove that there is a (1 + -g^j-)-approximation 
algorithm for finding a minimum cost spanning tree with maximum degree B. 
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5 

Matroids 



t 

In this chapter, we will study matroids, a combinatorial abstraction, which general- 
izes spanning trees and host of other combinatorial structures. After introducing matroids 
and stating some basic properties, we address the two most important polynomial-time 
solvable problems in this formalism: that of finding a maximum weight basis and of finding 
a maximum weight common independent set of two matroids (the so-called two-matroid in- 
tersection problem), and show integral characterizations using an iterative proof. We then 
consider the duals of these problems and prove a min-max theorem on matroid intersection. 

We extend the method developed for the exact characterizations to two NP-hard ma- 
troid optimization problems. First, we define a degree-bounded version of the minimum 
cost basis problem for matroids, and adapt our iterative proof method to supply additive 
approximation algorithms. Then, extending the proof for the two matroid intersection 
problem, we present a (k — l)-approximation algorithm for the unweighted k matroid in- 
tersection problem: finding a maximum cardinality common independent set in k matroids 
defined on the same ground set. 



5.1 Preliminaries 

Definition 5.1.1 A pair Ai = (S,I) is a matroid if I is a nonempty collection of subsets 
of S with the following properties. 

(i) G X. 

(ii) A G 1 and B C A Bel 

(iii) A, B G 1 and \B\ > \A\ =>• 3x G B \ A such that A U {x} G X. 

S is called the ground set of the matroid M. A set A C S is called independent if 

t © Copyright 2011 by Lap Chi Lau, R. Ravi and Mohit Singh. Published 2011 by Cambridge 
University Press. 
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A £ 1 else it is called dependent. A maximal set A G X is called a basis of M. Observe 
that Property (iii) implies that all bases have the the same cardinality. 

Examples of Matroids 

(i) Graphic Matroid: Given an undirected graph G = (V,E), the graphic matroid 
of G is denned as M.q = (E,Ig) where Xq = {F C E \ F contains no cycles}. 

(ii) Uniform Matroid: Given a set S and an integer k > 0, the uniform matroid of 
rank k is denned as M k s = {S,l k ) where l k = {T C S : \T\ < k}. 

(iii) Partition Matroid: Given a set S = tt)f =1 5i (ttl is the operation of disjoint union) 
and integers m, . . . ,n& > 0, the partition matroid is denned as M. = (S, I) where 
l={S k =l T i C # : \Ti\ <m}. 

(iv) Linear Matroid: Let Abe an mxn matrix and S = {1, . . . , n}. For any 1 < z < n, 
let A 1 denote the i*' l -column of A. The linear matroid over matrix A is denned as 
M-A = {S,Ta) where Za = {T C S : A 1 for i <E T are linearly independent}. 

(v) Matroid Restriction: Let M. = (S,Z) be a matroid and T C S. Then the matroid 
restriction of M. to the set T is the matroid Ai T = (T,It) where It = {R ■ R G 
I,_RC T}. 

It is quite straightforward to verify that the above examples satisfy the properties of 
matroids and we leave it as an exercise for the reader. 

Definition 5.1.2 (Rank function) Given a matroid M. = (S,I), the rank function 
r_A4 : 2 s — > Z of the matroid Ai is defined as rj^{T) = max[/cT,(7eX \ U\. 

We will drop the subscript M. from the rank function tm when the matroid M. is 
clear from the context. Observe that A G X if and only if r(A) = \A\. We also have 
the following important property about rank function of matroids. In the terminology 
introduced in Chapter 2, the property states that the rank function of matroids is (fully) 
submodular. 

Lemma 5.1.3 Let r be the rank function of matroid M. = (S,I). Then for all A, B C S, 
we have r(A) + r(B) > r(A n B) + r(A U B). 

Proof Let r(An B) = ki, r(A U B) = k 2 . This implies that 3V C A n B such that 
r(V) = \ V\ = k\. Similarly, there exists U C A U B such that r{U) = \U\ = k^. Moreover, 
since every independent set can be extended to a basis, we can assume that V C U. Since 
U is independent, we have r(A) > \U D A\ and r(B) > \U P\ B\. Now, we have 

\ur\A\ + \ur\B\ = \un {Aub)\ + \un (AnB)\ > \u\ + |v| 

^ r(A) + r(B) > r(A U B) + r(A n 5) 
as desired. □ 
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We now define two important operations on matroids. 

Definition 5.1.4 (Deletion) Given a matroid Ai = (S,I) and x £ S we define M\x = 
(S \ {x},!') where X' = {T \ {x} : T £ X} to be the matroid obtained by deleting x from 
M . The rank function of M \ x, denoted by r\, is related to the rank function r of M by 
the formula ri(T) = r(T) for T C S \ {x}. 

Definition 5.1.5 (Contraction) Given a matroid M. = (S,I) and x G S we define 
M/x = (S \ x,I") as the matroid obtained by contracting x in M where X" = {T C 
S \ {x} : T U {x} € X} if {x} is independent, and I" = X if {x} is not independent. The 
rank function of M/{x}, denoted by r2, is related to the rank function of M by the formula 
r 2 (T) = r(T U {x}) - r({x}) for T C S\x. 

Note that if {x} is not independent, then M/x = M \ x. 

5.2 Maximum Weight Basis 

We now study the maximum weight basis problem in a matroid, and give an integral 
characterization by an iterative argument. Given a matroid M = (S,I) and a weight 
function w : S — > M, the task is to find a basis of M of maximum weight. In the special 
case of graphic matroids in connected graphs, this problem reduces to the maximum weight 
spanning tree problem. If we set edge costs to be the negative of their weights, this is the 
same as the minimum cost spanning tree problem which we studied in Chapter 4. 

5.2.1 Linear Programming Formulation 

We begin by giving a linear programming formulation for the problem. Let x e denote the 
indicator variable for element e, with the intent that x e = 1 if e is in the solution and 
otherwise. We obtain the following linear programming relaxation LP mat {M) after relaxing 
the integrality constraints on the variables x. In the following we use the shorthand x(T) 
to denote X^eeT x e f° r an y subset T of the ground set S. 

maximize w e x e 

subject to X (S) = r(S) 

X (T) < r(T) yrcs 

x e > VeeS 

Solving the linear program. Observe that the above linear program is exponential in 
size and hence, an efficient separation routine is needed to separate over these constraints. 
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The separation routine needs to check that x(T) < r(T) for each T C S. Cunningham [40] 
provided such a separation routine which as an input uses the independence oracle for 
matroid M.. Since the rank function of a matroid is a submodular function, one can 
also uses an algorithm for minimizing submodular function [150] to separate over these 
constraints. 



5.2.2 Characterization of Extreme Point Solutions 

We now give a characterization of extreme point solutions of the linear program LP ma t(M) 
by showing that the linearly independent set of tight constraints associated with it can be 
chosen to form a special structure. 

Definition 5.2.1 (Chain) A subset C C 2 s is a chain if A G C, B G C, then A C B or 
BCA. 

We will use an uncrossing argument to show that the linearly independent set of 
tight constraints can be chosen to form a chain. Given an extreme point solution x to 
LP m at(M) let P = {T C S : x(T) = r(T)} be the set of tight constraints. For a set 
T C S, let xOO denote the characteristic vector in M) s \ that has an 1 corresponding to 
each element e G T and otherwise. We first show that P is closed under intersection and 
union. 

Lemma 5.2.2 If U,V G P, then both U CiV and U U V are in T . Furthermore, x(P) + 

x(v) = x(unv) + x (uuv). 

Proof 

r{U) + r(V) = x{U) + x(V) 

= x(unv) + x(uuv) 

< r(U n V) + r(U U V) 

< r(U) + r(V) 

The first equality is by the fact that U, V G P. The second equality follows from basic 
set properties. The third inequality follows from the constraints in the matroid basis 
LP m at(M). The last equality is because of the submodularity of the rank function r as 
shown in Lemma 5.1.3. Moreover, x(U)+x(V) = x(U ^ V) + x(U U V) follows from basic 
set properties. □ 

Lemma 5.2.3 If £ is a maximal chain subfamily of T, then span(C) = span(P). 

Proof The proof follows exactly the same argument as in Lemma 4.1.5. We show exactly 
where the argument differs and why we obtain a chain in this case while we could only 
argue a laminar structure in Lemma 4.1.5. Lemma 5.2.2 shows that two tight sets A and B 
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can always be uncrossed, not only when A and B intersect as was the case in Lemma 4.1.4. 
Hence, even if A, B are two tight sets and A n B = 0, we can uncross them and ensure 
that no such two sets exist among the constraints defining x. □ 

Observe that for the spanning tree formulation in Section 4.1, we obtained a laminar 
family characterizing the extreme point solution in Lemma 4.1.5. For the matroid basis 
problem, which is a generalization of spanning tree problem, Lemma 5.2.3 implies that the 
extreme point can be characterized by the chain which is a simpler structure than a general 
laminar family. This discrepancy can be explained by the formulation that we chose for 
the spanning tree problem. Indeed, applying uncrossing on the partition LP does give a 
characterization such that the extreme point is characterized by a chain, albeit over the 
edges and not the vertices of the graph (see exercises) . 

The following lemma follows from Lemma 5.2.3 and the Rank Lemma. 

Lemma 5.2.4 Let x be any extreme point solution to LP ma t{M) with x e > for each 
element e £ S. Then there exists a chain C such that 

(i) x(T) = r{T) for each T C £. 

(ii) The vectors in {x(T) : T 6 £■} are linearly independent. 
(hi) \C\ = \S\. 

5.2.3 Iterative Algorithm 

We now give an iterative algorithm which constructs an integral solution from the linear 
program and shows that the linear programming formulation is integral. The algorithm is 
shown in Figure 5.1. 

Iterative Maximum Weight Matroid Basis Algorithm 

(i) Initialization B 0. 

(ii) While B is not a basis do 

(a) Find an optimal extreme point solution x to LP mat (M) and delete every 
element e with x e = from M., i.e., M <— M \ e. 

(b) If there is an element e such that x e = 1, then update B<-BU {e} and 
set M <r- M/e. 

(iii) Return B. 



Fig. 5.1. Maximum Weight Matroid Basis Algorithm 



5.2.4 Correctness and Optimality 

First, we prove that the algorithm will terminate in the following lemma. 
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Lemma 5.2.5 For any extreme point solution x to LP ma t(M) with x e > for every 
element e, there exists an element e with x e = 1. 

Proof We present two proofs for this lemma. 

(Global Counting Argument.) Suppose for a contradiction < x e < 1 for each e £ S. 
Then the number of variables is exactly Since there is no element e with x e = 0, 
every tight inequality is of the form x(T) = r(T). By Lemma 5.2.4, there are \C\ linearly 
independent tight constraints of the form x(T) = r(T) where C is a chain. Since there is 
no integral x e , there is no singleton element in the chain. Thus every pair of consecutive 
sets in the chain differ by at least two elements, so \C\ < \S\/2 which is a contradiction. 

(Local Fractional Token Argument.) As in the case of spanning trees, there is a 
fractional token argument that can be generalized to the degree bounded case. For each 
e G S assign one token and distribute x e fraction of it to the smallest set T in the chain 
of tight constraints containing it. For every tight set T with child C, the set of fractional 
tokens assigned to it is x(T) — x(C) = r(T) — r(C) which cannot be zero (due to linear 
independence of this pair), and hence is at least one. The remaining (1 — x e ) tokens for 
any element e gives the desired contradiction. □ 

Next we show that the returned solution is a maximum weight basis, which is proved 
in the following theorem. 

Theorem 5.2.6 The Iterative Matroid Basis Algorithm returns a maximum weight basis 
in polynomial time. 

Proof This is proved by induction on the number of iterations of the algorithm. The base 
case is trivial to verify. Let M = (S,Z) denote the matroid in the current iteration. If the 
algorithm finds an element e with x e = we update the matroid to M \ e. Observe that x 
restricted to S \ {e}, say x', is a feasible solution to LP mat (M \ e). This is easily checked 
using the rank function of M \ e which is identical to the rank function of M on the sets 
not containing e. By induction, we find a basis B of M\e of weight at least w-x'. Observe 
that B is also a basis of M and costs at least w ■ x' = w ■ x. Hence, the induction claim is 
true in this case. 

Now, suppose the algorithm selects an element e with x e = 1. Then the algorithm 
updates the matroid M to M/e and B to B U {e}. Let r denote the rank function of 
M and r' denote the rank function of M/e. We now claim that x restricted to S \ {e}, 
say x', is a feasible solution to LP ma t(M/e). For any set T C S \ {x}, we have x'(T) = 
x(T U {e}) — x e = x(T U {e}) — 1 < r(T U {e}) — 1 = r'(T). By the induction hypothesis, 
we obtain a basis B' of M/e of weight at least w • x' . Then B' U {e} is a basis of M of 
weight at least w • x' + w e = w • x as required. This shows that the algorithm returns a 
maximum weight integral basis of the matroid M. □ 
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This also shows that the LP mat (M) is an exact formulation of the maximum weight 
basis problem. 

Theorem 5.2.7 The extreme point solutions of LP ma t{M) are bases of matroid M. 



5.3 Matroid Intersection 

Given matroids Mi = (S,I±) and M2 = (S, I2) and a weight function w : S — > R, the max- 
imum weight two matroid intersection problem is to find a set T C S of maximum weight 
which is independent in both M\ and M2, i.e, T is a maximizer of maxTcs,TeX 1 nX2 w ('^) = 
Y2eeT w e- This problem generalizes many important problems, including the maximum 
weight matching in bipartite graphs (Chapter 3) and maximum weight arborescence prob- 
lem (Chapter 6). 

Examples of Matroid Intersection 

(i) Matchings in Bipartite graph: Given a bipartite graph G = (AL)B,E), let 
Ma = (E,I±) be a partition matroid on E where X\ = {F C E \ oIf{v) < 1 Vd £ 
^4}. Similarly, let Mb = {E,T<i) be a partition matroid on E where X2 = {F C 
E I oIf(v) < 1 Vt> G B}. Observe that T G Xi n X2 if and only if T is a matching 
in G. Hence, finding a maximum weight matching in G is equivalent to finding a 
maximum weight independent set in the intersection of matroids Ma and Mb- 

(ii) Arborescence: Given a directed graph G = (D,A) with root r, let M\ = (A,X\) 
be the graphic matroid on the underlying undirected graph of G (where we ignore 
arc directions). Let M2 = (A ^2) be the partition matroid where I2 = {B C A : 
d l ^{v) < 1 \/v G D \ {r} and (r) = 0}. Observe that B is a common basis in X\ 
and X2 if and only if B is an arborescence rooted at r. 

We now show that an efficient procedure for the maximum weight basis in the inter- 
section of two matroids. In contrast, the matroid intersection problem for three or more 
matroids is NP-hard in general. 



5.3.1 Linear Programming Relaxation 

We now give a linear programming formulation for finding a maximum weight common 
independent set in the intersection of two matroids, and then give an iterative argument 
to show that it is integral. Let x e denote the indicator variable for element, with x e = 1 
if e is in the common independent set and otherwise. We obtain the following linear 
programming relaxation LPi nt (Mi, M2) after relaxing the integrality constraints on the 
variables x. Here rj(T) denotes the rank of the set T in the matroid Mj. 
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maximize 




subject to 



x{T) < n(T) 
x(T) < r 2 (T) 



VTCS 



VTC5 



x e > 



Solving the linear program. To get a separation oracle for minimum weight matroid 
basis problem, it suffices to separate over the inequalities in LPi nt {M\, M 2 ). This can be 
done if we are given as input independence oracles for each of the matroids M\ and M2, by 
using the work of Cunningham [40] as before, or any algorithm for minimizing submodular 
functions. 



We now give a characterization of extreme points of the linear program LPj nt (Mi, M2) 
by showing that the independent set of tight constraints can be chosen to form a union 
of two chains. The proof is quite straightforward and uses the characterization of tight 
inequalities for the matroid basis problem. 

Given an extreme point solution x to LPi nt (Mi, M2) let T\ = {T C S : x{T) = 
n(T)} and T 2 = {T C S : x(T) = r 2 (T)} be the set of tight constraints. 

Lemma 5.3.1 There exist two chains C\ and C2 such that span(C\ UC2) = span{F\ U T2) 
and constraints in sets C\ and C2 are linearly independent. 

Proof Applying Lemma 5.2.3 to families F\ and T2 separately, we obtain two chains 
and C' 2 such that span^) = span(F\) and span(C 2 ) = span (^2). Now, picking a maximal 
independent family from C[ U C' 2 gives us the desired chains. □ 

The following lemma follows from Lemma 5.3.1 and the Rank Lemma. 

Lemma 5.3.2 Let x be any extreme point solution to LPi n t(M\, M 2 ) with x e > for each 
element e G S. Then there exist two chains C\ and C2 such that 



5.3.2 Characterization of Extreme Point Solutions 



(i) x(T) = n(T) for each T C d for i = {1, 2}. 

(ii) The vectors in {x(T) : T G C±} U {x(T) : T G C2] are linearly independent. 
(hi) |Ci| + |C 2 | = |5|. 
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5.3.3 Iterative Algorithm 

We now give an iterative algorithm which constructs an integral solution from the linear 
program and shows that the linear programming formulation is integral. The algorithm is 
shown in Figure 5.2. 

Iterative Matroid Intersection Algorithm 

(i) Initialization / •<— 0. 

(ii) While S / do 

(a) Find an optimal extreme point solution x to LPi n t{M\, M 2 ) and delete 
every element e with x e = from M\ and M 2 , i.e., M\ 4- M\ \ e and 
M 2 <- M 2 \ e. 

(b) If there is an element e such that x e = 1 then and update I <— I U {e}, 
Mi <- Afi/e and M 2 4- M 2 /e. 

(iii) Return /. 



Fig. 5.2. Matroid Intersection Algorithm 



5.3.4 Correctness and Optimality 

First, we prove that the algorithm will terminate in the following lemma. 

Lemma 5.3.3 For any extreme point solution x to LPi n t(M\, M 2 ) with x e > for every 
element e, there exists an element e with x e = 1. 

Proof Suppose for a contradiction < x e < 1 for each e £ S. Then the number of variables 
is exactly \S\. Since there is no element e with x e = 0, every tight inequality is of the form 
x(T) = r\{T) or x(T) = r 2 (T) for some T C S. By Lemma 5.3.2, we obtain two chains 
C\ and C 2 defining x. We now show a contradiction to the fact that \S\ = \C\\ + \C 2 \ by a 
counting argument. 

We give two tokens to each element in S for a total of 2|5| tokens. Now, we collect 
two tokens for each member of C\ and C 2 and an extra token showing the contradiction. 
This is done as follows. Each element e assigns one token to the smallest set T\ £ C\ such 
that e £ Ti and the other token to the smallest set T 2 G C 2 such that e 6 T 2 . We now 
claim that each set in C\ U C 2 obtains at least two tokens. 

The argument is identical for sets in C\ and C 2 . Let T 6 C\ and R be the largest set 
in C\ such that R CT. Now, we have x(T) = ri(T) and x(R) = ri(R). Subtracting, we 
obtain x(T \R) = r\(T) — r±(R). If T \ R = then T = R and we have a contradiction to 
the linear independence of the constraints. Also, since x(T\R) is an integer and < x e < 1 
for all e, we have that \T \ R\ > 2. Thus T receives one token for each element in T \ R for 
a total of at least two tokens. Therefore, every set in C\ U C 2 receives at least two tokens. 
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Now, we show that there is at least one extra token. Observe that S G C\ or S 6 C2 but 
not both; say it is in C\. Hence, there exists an e such that e is not contained in any set 
in Ci. Hence, one token for e has not been used in the above counting argument, giving us 
the desired extra token for the contradiction. □ 

It remains to check that the returned solution is optimal. The proof of the following 
theorem is straightforward and follows from an argument identical to Theorem 5.2.6 (and 
is not duplicated). 

Theorem 5.3.4 The Iterative Matroid Intersection Algorithm returns a maximum weight 
set independent in the intersection of both matroids. 

This also shows that the LPi nt {M\, M2) is an exact formulation of the maximum 
weight matroid intersection problem. 

Theorem 5.3.5 The extreme point solutions of LPi n t{M\, M2) correspond to independent 
sets in the intersection of M\ and M<i- 



5.4 Duality and Min-Max Theorem 

In this section we consider the dual problems of maximum weight basis and maximum 
weight matroid intersection and show their integrality; we also prove a min-max theorem 
for matroid intersection. The former results rely on integrality results about covering linear 
programs where the constraint matrices are network matrices (Chapter 8). 



5.4-1 Dual of Maximum Weight Basis 

We first consider the dual linear program of the maximum weight basis problem and argue 
its integrality. The following linear program, which we call LPd ma t(M), is the dual linear 
program to LP mat (M). We use y(T) to denote the dual variable for a subset T, and we 
assume that the weights w e are nonnegative integers. 



minimize r(T)y(T) 

TCS 

subject to y(T) > w e Ve £ S 



T-.e&T 

y{T) > VT C S 

The uncrossing technique can be used to prove the following claim. 
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Claim 5.4.1 There is an optimal solution y to LP^ mat (M) with the set C = {T C S : 
y(T) > 0} being a chain of S. 

Proof Pick an optimal solution that minimizes the number of pairs of sets that are either 
intersecting or disjoint in the support. We will prove that this number is zero, and this 
would imply the claim. Suppose for a contradiction that this number is not zero and A 
and B are such a pair with min{y(A),y(B)} = e. Consider the alternate solution y' with 
y'(A) ^ y(A)-e,y'(B) ^ y(B)-e,y'(AuB) ^ y(AuB) + e and y'(AnB) «- y(AnB)+e, 
with y'(S) <— y(S) for all other sets S. It is easy to see that y' is also a feasible solution 
to LP^ mat (M). Also, by the submodularity of the rank function (Lemma 5.1.3), we have 

e(r(A) + r(B)) > e(r(A UB) + r(A n B)), 

and thus the objective function value does not increase. Finally, the new solution can 
easily be checked to have fewer pairs that are disjoint or intersecting in the support, giving 
the contradiction and finishing the proof. □ 

Claim 5.4.1 implies that the following restricted linear program, denoted by LP r d m at(M), 
has the same objective value as LPdmat(M). 

> w e Ve £ S 

> VTGC 

As C is a chain, one can show that the constrained matrix of LP r d ma t(M) is a network 
matrix: see Section 8.4.1 in Chapter 8. Then, by the results on LP's with network matrix 
constraints (Theorem 8.1.1), it follows that LP r d ma t(M) is integral. Since we have shown 
that LPdmat(M) always has integral optimal solutions, we also get the following result. 

Theorem 5.4.2 The linear programming formulation LPd ma t(M) is integral. 



5.4-2 Dual of Two Matroid Intersection 

Next we prove the integrality of the dual linear program of matroid intersection, and 
obtain a min-max theorem on matroid intersection. The following linear program, denoted 
by LPdint(Mi, M2), is the dual linear program to LPi nt {M\, M2) in Section 5.3. 



minimize r{T)y(T) 
Tec 

subject to 

TeC:eST 

y(T) 



77 



minimize Yl n{T) yi {T) + r 2 {T)y 2 {T) 

TCS 

subject to Y^ (T) + y 2 {T)^ > w e Ve £ S 

T:eeT 

Vi{T) > VTCS,l<i<2 

The following claim follows by the same uncrossing argument as in Claim 5.4.1. 

Claim 5.4.3 There is an optimal solution y to LPdi n t{Mi, M 2 ) where the sets C\ = {T C 
S : yi (T) > 0} and C 2 = {T C S : i/a(T) > 0} are frotfi ckms 0/ S. 

As in the case of the dual of maximum weight basis, Claim 5.4.3 implies that the 
following restricted linear program, denoted by LP r di n t(Mi, M 2 ), has the same objective 
value as LP dint {M 1 ,M 2 ). 

minimize ^ r^y^T) + ^ r 2 (T)y 2 {T) 

TeCi Tec 2 

subject to Y yi( T )+ Y V2( T ) > w e VeeS 

TeCi:eST TeC 2 :e6T 

yi (T) > VT e Ci, 1 < i < 2 

As Ci and C2 are chains, one can show that the constrained matrix of LP r( n n t(Mi, M 2 ) 
is a network matrix; see Section 8.4.2 in Chapter 8. Then, by the result on network matrix 
(Theorem 8.1.1), it follows that LP r< a nt {M\^M 2 ) is integral. As before, since we have 
shown that LPdi n t(M) always has integral optimal solutions, we also get the following 
result. 



Theorem 5.4.4 The linear programming formulation LPdi nt (Mi, M 2 ) is integral. 

Using Theorem 5.3.5 and Theorem 5.4.4, we can obtain the following min-max the- 
orem on the maximum cardinality of a common independent set in two matroids M\ and 
M 2 . 

Theorem 5.4.5 Given matroids M\ = (S,I±) and M 2 = (S,Z 2 ) with rank functions r\ 
and r 2 respectively, the size of a maximum common independent set in X\ and X 2 is given 
by 

max 1/1 = minfri (T) + r 2 (S \T)) 
/€Zinx 2 tcs 

Proof The problem of finding a maximum cardinality common independent set is a special 
case of the maximum weight two matroid intersection problem when w e = 1 for all e £ S. 
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By the strong duality theorem of linear programming (Theorem 2.1.9), the objective value 
of LPi n t(Mi, M2) is the same as the objective value of LP,n nt {M\,M'i). For the primal 
program LPi n t(M±, M2), observe that x e < r({e}) < 1 since r is the rank function of 
a matroid. By Theorem 5.3.5, LPi nt {M\, M2) is integral and hence there is an optimal 
solution with x e G {0, 1}, and thus the optimum value of LPi nt {M\ : M2) = max; 6 i in i 2 |/|. 

For the dual program LP^ int {M\, M2), since w e = 1 for all e G S and ri{T) > for 
all T C S, we can assume that yi(T) < 1 for all T C S. By Theorem 5.4.4, LP dint (Mi, M 2 ) 
is integral and hence there is an optimal solution with Ui(T) G {0, 1} for all T C S and 
i = {1,2}. By Claim 5.4.3, we can assume that the set C\ = {T C S : yi(T) = 1} is a 
chain. Furthermore, since w e = 1 for all e G S and rj(T) > for all T C 5, we can assume 
that there is only one set T with yi(T) = 1, with T being the largest set in the chain 
C\. Similarly, we can assume that there is only one set T' with y2(T') = 1. Since w e = 1 
for all e G S 1 and y is a feasible solution to LP^ ni (Mi, M2), we must have TUT' = S. 
As the rank function r of a matroid is a monotone function, we can further assume that 
T' = S \ T, and thus there is an optimal solution to LP dint {M\, M2) with objective value 
ri(T) + V2(S \ T) for some T C S, proving the theorem. □ 



5.5 Minimum Bounded Degree Matroid Basis 

In this section we consider the minimum bounded degree matroid basis problem, a gener- 
alization of the minimum bounded degree spanning tree problem. We are given a matroid 
M = (S,Z), a cost function c on the ground set S, a hypergraph H = (S, E) on the same 
ground set, and an upper bound g(e) for each hyperedge e G E(H). The task is to find a 
basis B of minimum cost such that \B PI e| < g(e) for each hyperedge e G E(H). 

One motivation for considering the matroid generalization was the following problem 
posed by Frieze [59]: "Given a binary matroid M4 over the columns of a 0, 1-matrix A and 
bounds gi for each row i of A, find a basis B of matroid Ma such that there are at most 
gi ones in row i (for all rows i) among the columns in B". The main result of this section 
is the following theorem. 

Theorem 5.5.1 There is a polynomial time algorithm for the minimum bounded degree 
matroid basis problem which returns a basis B of cost at most OPT such that \B D e\ < 
g(e) + A — 1 for each e G E(H). Here A = max„ g g |{e G E(H) : v G e}\ is the maximum 
degree of the hypergraph H and OPT is the cost of an optimal solution which satisfies all 
the degree constraints. 

When the hyperedges in H are disjoint, the maximum degree A is 1 and the above 
theorem gives an optimal result. Note that this also follows from the matroid intersection 
theorem: when the hyperedges induce disjoint sets of the ground set, they define a par- 
tition matroid, and the solution we find is a minimum cost common basis of the original 
matroid and this partition matroid. We mention more applications of Theorem 5.5.1 to 
approximation algorithms in the last part of this section. 
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5.5.1 Linear Programming Relaxation 

Let r : 2 s — > Z + denote the rank function of matroid M. We have already argued in 
Lemma 5.1.3 that r is a monotone submodular function. We denote the following relaxation 
as LP mat (M, H). 



= r{S) 

< r{T) VT C S 

< g{e) VeeE{H) 

x v < i y v es 

This linear program is exponential in size but can be separated over in polynomial 
time as before if given access to an independent set oracle for the underlying matroid. 

5.5.2 Characterization of Extreme Point Solutions 

The following lemma follows from Lemma 5.2.4 and the Rank Lemma. 

Lemma 5.5.2 Let x be any extreme point solution to LP ma t(M, H) with x e > for each 
element e G S. Then there exists a set R C E and a chain C such that x is the unique 
solution to the following linear system. 

(i) x(T) = r(T) for each T G C and x(e) = g(e) for each e G R. 

(ii) The vectors in {x(T) : T G £} U {x( e ) '■ e £ R} are linearly independent. 
(hi) \S\ = \C\ + \R\. 

5.5.3 Iterative Algorithm 

The iterative algorithm is similar to that of the minimum bounded degree spanning tree 
problem in Chapter 4. 

5.5.4 Correctness and Performance Guarantee 

The degree constraint is only violated by at most A — 1 in Step 2(c) of the algorithm. Theo- 
rem 5.5.1 follows by an inductive argument if the algorithm always terminates successfully, 
which we will prove in the following lemma. 

Lemma 5.5.3 An extreme point solution x to LP ma t(M, H) must satisfy one of the fol- 
lowing. 



minimize ^ 

c v x v 

ves 

subject to X (S) 
x(T) 
x(e) 
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Iterative Minimum Bounded Degree Matroid Basis Algorithm 

(i) Initialization B 0, 

(ii) While B is not a basis do 

(a) Find an optimal extreme point solution x to LP mat (M, H). Delete v with 
x v = 0. Update each edge e G E(H) with e <— e \ {u }. Update matroid 
M <- M \ v. 

(b) For each element v with x„ = 1, include v in B and decrease g(e) by 1 for 
each eB v. Update M <- M/v. 

(c) For every e G E(H) with |e| < y(e) + A — 1, remove e from E(H). 

(iii) Return B. 

Fig. 5.3. Minimum Bounded Degree Matroid Basis Algorithm 

(i) There is an element v with x v = 1. 

(ii) There is a hyperedge e such that \e\ < g(e) + A — 1. 

Proof The proof is by a local fractional token argument. Each element is initially assigned 
A tokens, for a total of A • \S\ tokens. For each element v, 1 — x v token is redistributed to 
each hyperedge that contains v, and A • x v token is redistributed to the smallest set T G C 
which contains e. This is possible since each element is contained in at most A hyperedges. 
We shall show that if neither of the above conditions are satisfied, then each set in £ and 
each hyperedge constraint in R can collect A tokens, and there are still some tokens left. 
This would imply \S\ > \C\ + \R\, which contradicts to Lemma 5.5.2. 
For each hyperedge e in R, it collects 

^(1 - x v ) = \e\-^2x v = |e| - g(e) > A 

tokens. The second equality follows because e is tight, and the last inequality follows 
because the second condition in the lemma is not satisfied. This shows that each hyperedge 
constraint in R can collect at least A tokens. 

For each T G £, let U G C be its child in the chain C Then T collects 

A • (x(T) - x(U)) = A • (r(T) - r(U)) > A • 1 = A 

tokens. The inequality follows because x(T) and x(^) are linearly independent and r is 
an integer monotone function. This shows that each set in £ can collect at least A tokens. 

It remains to show that there are some unused tokens. If some element is not in 
exactly A hyperedges in R or if S ^ £, then there are some tokens left, which contradicts 
that \S\ = \£\ + \R\. Otherwise, we have A • x(S') = S eg RX( e ) 5 which contradicts the 
linear independence of the characteristic vectors of the sets in R and S G C. In either case 
there is a contradiction implying that one of the conditions in the lemma must be satisfied. 

□ 
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5.5.5 Applications 

We highlight some applications of the bounded degree matroid basis result. 

Minimum Crossing Spanning Tree 

In the minimum crossing spanning tree problem, we are given a graph G = (V, E) with edge 
cost function c, a collection of cuts (edge subsets) C = {C±, . . . , C m } and bound gi for each 
cut Cj. The task is to find a spanning tree T of minimum cost such that T contains at most 
gi edges from cut Cj. The minimum bounded degree spanning tree problem is the special 
case where C = {5(v) : v £ V}; note that A = 2 for the minimum bounded degree spanning 
tree problem. The following result can be obtained as a corollary of Theorem 5.5.1. 

Corollary 5.5.4 There exists a polynomial time algorithm for the minimum crossing span- 
ning tree problem that returns a tree T with cost at most OPT, and such that T contains at 
most gi + d — 1 edges from cut Ci for each i, where d = max eg £ \{C\ : e £ Cj}| and OPT is 
the cost of an optimal solution which satisfies all the cut degree constraints. 

Proof Let M = (E,Z) denote the graphic matroid over the graph G. The hypergraph 
H is denned with V{H) = E(G) and E{H) = {d : 1 < i < m}. Note that A = 
maXj, e y(^) \{e £ E{H) : v £ e}\ = max egE (G) |{Cj : e 6 Cj}| = d. So, using Theorem 5.5.1, 
we obtain a basis T of matroid M (which is a spanning tree), such that |TnQ| < gi + d—1. 

□ 

Minimum Bounded- Ones Binary Matroid Basis 

For the minimum bounded-ones binary matroid basis problem, we are given a binary 
matroid over the columns of a 0, 1-matrix A and bounds gi for each row i of A. The 
task is to find a minimum cost basis B of matroid Ma such that there are at most gi ones 
in row i (for all rows i) among the columns in B. The following result is obtained as a 
corollary of Theorem 5.5.1. 

Corollary 5.5.5 There exists a polynomial time algorithm for the minimum bounded-ones 
binary matroid basis problem which returns a basis B of cost at most OPT, such that there 
are at most gi + d — 1 ones in any row restricted to columns of B. Here d is the maximum 
number of ones in any column of A, and OPT is the cost of an optimal solution satisfying 
all the row constraints. 

Proof Let M = Ma be a linear matroid and define a hypergraph H where the vertex set 
is the columns of A. The hyperedges correspond to rows of A where = {A 3 : A^ = 1} 
where A J is the j th column of A. Note that A = max„ g y(^) |{e G E(H) : v € e}| = 
maxj \ {i : = 1}| = d, which is the maximum number of ones in any column of A. So, 
using Theorem 5.5.1, we obtain a basis of M = Ma such that number of ones in any row 
is at most gi + d — 1. □ 
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Minimum Bounded Degree Spanning Tree Union 



In the minimum bounded degree spanning tree union problem, we are given a graph G = 
(V, E) with edge cost function c, a positive integer k, and a degree upper bound g(v) for 
each vertex v. The task is to find a subgraph F which is the union of k edge-disjoint 
spanning trees and the degree of v in F is at most g(v). The minimum bounded degree 
spanning tree problem is a special case when k = 1. Theorem 5.5.1 implies the following 
result, which is optimal in terms of the degree upper bounds. 

Corollary 5.5.6 There is a polynomial time algorithm for the minimum bounded degree 
spanning tree union problem which returns a subgraph F of cost at most OPT which is the 
union of k edge-disjoint spanning trees and the degree of v in F is at most g(v) + 1. Here 
OPT is the cost of an optimal solution which satisfies all the degree upper bounds. 

Proof Let M = (E,Z) denote the union of k graphic matroids over the graph G, which 
is a matroid by the matroid union theorem (see e.g., [150]). The hypergraph H is defined 
with V(H) = E{G) and E{H) = {5(v) : v G V(G)}. Note that A = max vGV{H) \{e G 
E(H) : v G e}| = max eeB(G) \{5(v) : v G V(G) A e G S(v)}\ = 2. So, using Theorem 5.5.1, 
we obtain a basis F of matroid M (which is the union of k edge-disjoint spanning trees), 



Given k matroids Mi = (S,Ii), Mi = {S,T-i), . . . , = (S,Ik) on the same ground set S, 
the maximum k matroid intersection problem is to find a set T C S of maximum cardinality 
which is independent in all matroids M±, M2, . ■ ■ , M^. This problem is NP-hard already 
for k = 3 (see the exercises). We will present a 2-approximation algorithm for this problem 
when k = 3, and leave the generalization to general k to the exercises. 



The linear programming relaxation, denoted by LP^ nt {Mi, M2, M3), for three-matroid 
intersection is a natural extension of LPi nt (Mi, M2) for two-matroid intersection. Notice 
that we only consider the unweighted problem where w e = 1 for all e G S. 



such that \F n S(v)\ < g(v) + 1. 



□ 



5.6 k Matroid Intersection 



5.6.1 Linear Programming Relaxation 



maximize 




subject to 



x(T) < n(T) 
x(T) < r 2 (T) 
x(T) < r 3 (T) 



VTC5 



VTC5 



VT C S 



x e > 



Ve G S 
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There is an efficient separation oracle for this exponential-size linear program, as in 
the case for two-matroid intersection. 



5.6.2 Characterization of Extreme Point Solutions 

The proof of the following characterization follows the same lines as the proof of Lemma 5.3.2 
for two-matroid intersection. 

Lemma 5.6.1 Let x be any extreme point solution to LPsi n t(M\, M 2 , M 3 ) with x e > for 
each element e G S. Then there exist three chains Ci,C 2 ,C 3 such that 

(i) x(T) = n(T) for each T C d for i = {1, 2, 3}. 

(ii) The vectors in {%{T) : T G d} U {x(T) : T G C 2 } U {x(T) : T G C 3 } are ZinearZj/ 
mc?epenc?enl 

(iii) |Ci| + |C 2 | + |C 3 | = |5|. 



5.6.3 Iterative Algorithm 

The iterative algorithm is similar to that of two-matroid intersection, except that an ele- 
ment e with x e >\ will be chosen. 



Iterative Three Matroid Intersection Algorithm 

(i) Initialization 7 0. 

(ii) While S + do 

(a) Find an optimal extreme point solution x to LP^i nt {M\, M 2 , M3). Delete 
every element e with x e = from Mi, M 2 , M3, i.e., Mi <— Mi \ e and 
M 2 <- M 2 \ e and M 3 M 3 \ e. 

(b) If there is an element e with x e > \, then update 7 7U{e}, Mi <— M\/e 
and M 2 <- M 2 /e and M 3 M 3 /e. 

(iii) Return 7. 



Fig. 5.4. Three Matroid Intersection Algorithm 



5.6.4 Correctness and Performance Guarantee 

We first show that the iterative algorithm makes progress in each iteration. We then show 
that the algorithm returns a 2-approximate solution assuming it makes progress in each 
step. The proof of the former claim is similar to the proof of Lemma 5.3.3, while the proof 
of the later claim uses some basic results in matroid theory. 
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Lemma 5.6.2 For any extreme point solution x to LP^i n t(Mi, M2, M3) with x e > for 
every element e, there exists an element e with x e > \. 

Proof Suppose for a contradiction < x e < \ for each e G S. Then the number of 
variables is exactly By Lemma 5.6.1, we obtain three chains £1,(^2, C3 defining x. We 
now show a contradiction to the fact that |5| = \C\ \ + IC2I + \Cs\ by a counting argument. 

We give three tokens to each element in S for a total of 3|5| tokens. Now, we collect 
three tokens for each member of Ci, £2,(^3 and an extra token showing the contradiction. 
This is done as follows. Each element e assigns one token to the smallest set T{ G Ci such 
that e G Tj for i = {1, 2, 3}. We now claim that each set in C\ Li C2 U C3 obtains at least 
three tokens. 

The argument is identical for sets in C\,C2,Cs. Let T £ C\ and R be the largest set 
in C\ such that R CT. Now, we have x(T) = r\(T) and = ri(R). Subtracting, we 

obtain x(T \R) = r\{T) — r\(R). If T \ R = then T = R and we have a contradiction to 
the linear independence of the constraints. Also, since x(T\R) is an integer and < x e < \ 
for all e, we have that \T \ R\ > 3. Thus T receives one token for each element in T \ R for 
a total of at least three tokens. Therefore, every set in C\ Li C2 U C3 receives at least three 
tokens. Now, we show that there is at least one extra token. Observe that the whole set 
S can be in at most one of the three colelctions C\,C2 or C3, say it is in C\. Hence, there 
exists an e such that e is not contained in any set in 62- Hence, at least one token for e 
has not been used in the above counting argument, giving us the desired extra token for 
the contradiction. □ 

Theorem 5.6.3 The algorithm in Figure 5.4 returns a 2- approximate solution to the max- 
imum three- matroid intersection problem in polynomial time. 

Proof This is proved by induction on the number of iterations of the algorithm. The 
case when the algorithm finds an element e with x e = is handled as in the proof of 
Theorem 5.2.6. We focus on the case when the algorithm selects an element e with x e >\. 
In this case the algorithm updates the matroid M, to Mj/e and I to I Li {e}. Let w(x) be 
the objective value of the solution x in the current iteration. To prove the performance 
guarantee, it suffices to prove that there is a feasible solution in the next iteration with 
objective value at least w(x) — 2. Since we add one element to / and the objective value 
decreases by at most two, by a standard inductive argument we can prove that the returned 
independent set has size at least half the objective value of LP^i nt {M\, M2, M3), and thus 
the theorem follows. 

To prove the claim, we need to demonstrate a feasible solution in the next iteration 
with objective value at least w(x) — 2, after we select the element e and update the matroids 
Mi to Mi/e. Consider the solution x restricted to 5\e, denoted by x' . Note that x' has ob- 
jective value w (x) —x e , but it may not be a feasible solution to LP^i nt {Mi/ 'e, M2/e, M^/e), 
the linear program in the next iteration. In the next paragraph we will show how to mod- 
ify x' to satisfy all the constraints defined by matroid Mj/e, by decreasing the objective 
value by at most 1 — x e . By performing this modification to each of the three matroids, 
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we will have a feasible solution to LP^i nt {Mi/e,M2/e,M^/e) with objective value at least 
w(x) — x e — 3(1 — x e ) = w(x) — 3 + 2x e > w(x) — 2 since x e > ^, as desired. 

It remains to show how to modify the solution x' to satisfy all the constraints defined 
by Mj/e, while decreasing the objective value by at most 1— x e - Since x is a feasible solution 
to LPzi n t(Mi, M2, M3), it is obviously a feasible solution to LP ma t(Mi), the independent 
set polytope of matroid Mj. Since the independent set polytope of a matroid is integral, 
the solution x can be written as a convex combination of independent sets in Mj, i.e. 
x = Ylf=i ^jX(Ij) f° r some N, where Ylf=i^j = 1 f° r nonnegative Aj's and Ij is an 
independent set of Mj for each j. Assume that e ^ Ij for 1 < j < N' and e 6 Ij for 
N' < j < N. Then, by definition, Ylf=i ^« = 1 — x e- For each 1 < j < N', let fj / e be an 
element in the unique circuit (if exists) in Ij U {e}. Since Ij + e — /j is an independent set 
in Mj, it follows by definition that Ij — fj is an independent set in Mi/e. Similarly, Ij — e 
is an independent set in Mj/e for N' < j < N. Thus 

x* = \\x{h - fi) + • • • + \n'X(In> - !n>) + \n'+ix{In'+i - e) + . . . + \nx(In - e) 

is a feasible solution to LP mat (Mi/e), since it is a convex combination of independent sets 
in Mj/e. Furthermore w(x*) > w(x') — J2jLi = w(x') — (1 — x e ), proving the theorem. 

□ 



Notes 

Matroids were introduced by Whitney in the 1930's and a comprehensive review of related 
concepts that led to this as well as the rich literature on it is reviewed by Schrijver in his 
book [150]. The work of Edmonds [47, 48] first showed the polyhedral characterization 
results we presented in this chapter. The result for the minimum bounded degree matroid 
basis problem is by Kiraly, Lau and Singh [97]. 

Exercises 

5.1 Show that the five examples of matroids in Section 5.1 are indeed matroids by 
verifying the three matroid properties for each of them. 

5.2 Verify that the deletion and contraction matroids defined in Section 5.1 are indeed 
matroids by checking the two properties of matroids for them. Also prove that their 
rank function is correctly defined. Finally, convince yourself that for any element 
x of the matroid, if {x} is not independent, then M\x = M/x. 

5.3 Given a matroid M = (S,I) with rank function r, let M* = (S,Z*) be the dual 
matroid where X* = {I C S : S \ I contains a basis of M}. 

(a) Show that M* is a matroid. 

(b) Show that the rank r* of a set T in the dual matroid M* is given by r* (T) = 
\T\-r(S) + r(S\T). 
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5.4 Let M = (S,I) be a matroid with rank function r and let T C S. Matroid 
M T = (T,1 T ) is denned by I T = {R n T : R G X}. 

(a) Show that My is a matroid. 

(b) What is the rank function rx of My? 

5.5 Given two matroid M 1 = (S,li) and matroid M 2 = (S,X 2 ), let M 12 = (Si U 
S 2 ,Ii V X2) where S± and S2 are two copies of S such that S\ (1 S 2 = (/) and 
X\\l X 2 = {A U B : i e Ii, B e X 2 }- Let M' = (Si U S 2 ,T) be the partition 
matroid such that I El' if and only if / contains at most one copy of each element 
in S (remember there are two copies of each element, one in Si and the other in 
S 2 ). 

(a) Show that Mi 2 is a matroid. 

(b) Show that matroid union Mi U M 2 is isomorphic to matroid intersection 
M12 n M'. 

(c) Derive the rank function of matroid union of two matroids using the size 
of maximum size of an independent set in the intersection of these two 
matroids. 

5.6 Let D = (V, A) be a directed graph and subsets U, S of V. For X,Y CV, call X 
linked to Y if \X\ = \Y\ and D has |X|-vertex disjoint X — Y paths. (X is the set 
of starting vertices of these paths and Y is the set of ending vertices). 

Let X be the collection of subsets I of S such that some subset of U linked 
to /. Prove that M = (S,X) is a matroid. 

5.7 Show that finding a maximum cardinality independent set in the intersection of 
three matroids is NP-hard. 

5.8 Apply uncrossing to the partition LP for the spanning tree problem to obtain 
a characterization such that characteristic vectors for edge sets corresponding to 
linearly independent tight constraints form a chain. 

5.9 Consider the degree bounded minimum cost spanning tree problem where the de- 
gree bounds are imposed only on an independent subset of the nodes of the un- 
derlying graph. Show that this problem can be solved optimally in polynomial 
time. 

5.10 Show that Theorem 5.6.3 can be generalized to give a (k — l)-approximation algo- 
rithm for the maximum k matroid intersection problem. 
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6 

Arborescence and Rooted Connectivity 



t 

In this chapter we study problems in directed graphs and see how the techniques 
developed in previous chapters generalize to problems on directed graphs. We first con- 
sider exact formulations for the arborescence problem and a vertex connectivity problem 
in directed graphs. For the latter, we demonstrate the iterative method in the more so- 
phisticated uncrossing context which is applied to bi-set families instead of set families 
as in previous chapters. We then extend these results to degree bounded variants of the 
problems and use the iterative method to obtain bi-criteria results unlike previous chapters 
where the algorithm would be optimal on the cost and only violate the degree constraints. 

Given a directed graph D = (V, A) and a root vertex r € V , a spanning r-arborescence 
is a subgraph of D so that there is a directed path from r to every vertex in V — r. The min- 
imum spanning arborescence problem is to find a spanning r-arborescence with minimum 
total cost. We will show an integral characterization using iterative proofs, and extend 
this result in two directions. Given a directed graph D and a root vertex r, a rooted 
/c-connected subgraph is a subgraph of D so that there are k internally vertex- disjoint di- 
rected paths from r to every vertex in V — r. The minimum rooted fc-connected subgraph 
problem is to find a rooted fe-connected subgraph with minimum total cost. We extend 
the proofs in the minimum arborescence problem to show an integral characterization in 
this more general setting. 

As in the previous chapters, we extend the method developed for the exact charac- 
terization to the degree-bounded version of the problem. However, unlike in the undirected 
case, the resulting approximation algorithms have a multiplicative guarantee (rather than 
an additive guarantee) on the degree and the cost. We also show how additive guarantees 
on the degree can be recovered in the case of unweighted arborescence where arc costs are 
not considered. 



t © Copyright 2011 by Lap Chi Lau, R. Ravi and Mohit Singh. Published 2011 by Cambridge 
University Press. 
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6.1 Minimum Cost Arborescence 



This problem is studied using the same framework as before: first formulate the problem 
as a linear program, then characterize the extreme point solutions of this linear program, 
and finally use an iterative algorithm to find a minimum spanning r-arborescence. 



6.1.1 Linear Programming Relaxation 

It is easy to formulate the minimum spanning arborescence problem as a linear program. 
There is a natural choice that we call the directed LP, and this is an exact formulation. 
The directed LP requires that there is a directed path from the root r to every vertex in 
V — r. Or equivalently, by Menger's theorem (Theorem 2.2.1), the formulation requires 
that there is at least one arc entering every set which does not contain the root vertex. 
This directed LP formulation, originally due to Edmonds [46], for the minimum spanning 
arborescence problem is shown below. 

Just like earlier formulations, c a is the cost of choosing arc a, and x a is a binary 
variable to denote whether arc a is chosen in the arborescence or not. For a set S C V, 
the corresponding inequality x(5 in (S)) > 1 relates to a vector in M} A \; the vector has an 
1 corresponding to each arc a £ S m (S), and otherwise (recall that S m (S) is the set of 
all arcs incoming to a vertex v G S from any vertex outside S). This vector is called 
the characteristic vector of 8 m (S), and is denoted by x(<5 m (,S)). The term x(S m (S)) just 
means the sum Y. a e8 in (S) x <>- 

minimize ^ 

subject to x(S in (S)) > 1 VSC^-r 

x a > Vaei 

The above formulation actually captures all subgraphs containing an arborescence or equiv- 
alently, it is the up-hull of the arborescence polytope. The arborescence polytope can be 
obtained by adding the equality YlaeA x a = 1^1 ~~ 1- 

Although the number of constraints is exponential in the number of vertices, the 
availability of an efficient separation oracle ensures the polynomial-time solvability of this 
LP. Given any solution x, the separation oracle first constructs a graph with arc capacities 
as x a . It then computes the minimum-cuts from the root vertex r to every other vertex. 
If all the minimum-cuts have value at least 1, it is easy to see that the solution is feasible. 
If there exists a minimum-cut of value less than 1, the violated constraint is precisely the 
set of vertices that this cut separates. 

One can also write a compact formulation for the directed LP, using the equivalence 
of flows and cuts. This compact formulation provides an alternative way to solve the 
directed LP in polynomial time. For each vertex v € V — r, there is a variable for 
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each arc a, representing the flow value from r to v through the arc a. The proof of the 
equivalence of the linear programs is deferred to the exercises. 



minimize 


^ ^ c a x a 










aeA 








subject to 


fa 
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V v G V — r, 




ae8 tn (v) 








XT f a 


XT f a 
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E fa 
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6.1.2 Characterization of Extreme Point Solutions 

As in minimum spanning trees, the uncrossing technique is used to find a good set of 
tight inequalities that defines an extreme point solution to the directed LP. Let T = 
{S | x(5 m (S)) = 1} be the family of tight inequalities for an extreme point solution x 
in the directed LP. The next claim follows from the submodularity of the d in (S) function 
noted earlier in Proposition 2.3.9; see also Figure 2.3. 

Proposition 6.1.1 For X,Y C V, 

x{5 in {X)) + x{5 in {Y)) > x(5 in (X U Y)) + x{5 in {X n Y)), 
and equality holds if and only if E(X \Y,Y \ X) = 0. 

The following lemma shows that the family T is closed under intersection and union. 

Lemma 6.1.2 If S, T G T and SdT / 0, then both SdT and SUT are in T . Furthermore, 
X (S m (S)) + X (S in (T)) = X (S m (S n T)) + x($ m (S U T)). 

Proof 

1 + 1 = x(S in (S)) + x(S m (T)) 

> i(r(5nT)) + i(r(5uT)) 

> i + i 
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The equality follows from the fact that S, T € T . The first inequality follows from Propo- 
sition 6.1.1. The second inequality follows from the constraints for S PlT and SUT in the 
directed LP. Equality must hold everywhere and thus x(S in (S n T)) + x(5 in (S U T)) = 2. 
Therefore we must have equality for constraints for SCiT and SUT, i.e., x(5 in (S P\T)) = 1 
and x(5 in (SUT)) = 1, which imply that SflT and 5UT are also in T. Moreover, equality 
holds for Proposition 6.1.1 and thus E(S \T,T \ S) = and x($ in (S)) + x{$ in (T)) = 
X (5 in (SnT)) + x(S in (SuT)). □ 

Denote by span(F) the vector space generated by the set of characteristic vectors 
{x($ m (S)) I S G J 7 }. The following lemma says that an extreme point solution is char- 
acterized by tight inequalities whose corresponding sets form a laminar family. The proof 
follows the same steps as in the case of undirected spanning trees (see Lemma 4.1.5) and 
we do not duplicate it here. 

Lemma 6.1.3 If £ is a maximal laminar subfamily of T , then span(C) = span(J-). 
Lemma 6.1.3 and the Rank Lemma imply the following. 

Corollary 6.1.4 Let x be any extreme point solution to the directed LP. Then there exists 
a laminar family C such that 

(i) x(5 in (S)) = 1 for all S € C. 

(ii) The vectors in {x{$ m {S)) ■ S G £} are linearly independent. 

(iii) \A\ = 

A laminar family C defines naturally a forest L as follows (see Figure 6.1): Each node 
of L corresponds to a set in £, and there is an edge from set R to set S if R is the smallest 
set containing S. R is called the parent of S, and S is called the child of R. A node with 
no parent is called a root, and a node with no children is called a leaf. Given a node R, 
the subtree rooted at R consists of R and all its descendants. The forest L corresponding 
to the laminar family C will be used to perform the token counting arguments inductively. 



6.1.3 Iterative Algorithm 

We present an iterative algorithm to find a minimum spanning arborescence from an op- 
timal extreme point solution to the directed LP. The algorithm is similar to the Iterative 
1-edge-finding Algorithm in Figure 4.5 of Chapter 4 for finding minimum spanning trees. 
Again, we use the notation D/{uv} to denote the digraph obtained from D by contracting 
the arc uv. 
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(a) Laminar Set Family (b) Corresponding Forest 



Fig. 6.1. A laminar family and its corresponding forest. 

Iterative Minimum Spanning Arborescence Algorithm 

(i) Initialization F «— 0. 

(ii) While \V(D)\ > 2 do 

(a) Find an optimal extreme point solution x to the directed LP and remove 
every arc a with x a = from D. 

(b) Find an arc a = uv with x a = 1, and update F <— FU {a}, D <— D/{uv}. 

(iii) Return F. 

Fig. 6.2. Minimum Arborescence Algorithm 

6.I.4 Correctness and Optimality 

As in minimum spanning trees, assuming the algorithm terminates successfully, it is easy 
to show that the returned solution is a minimum spanning arborescence. The proof of the 
following result is identical to that of Theorem 4.1.10. 

Theorem 6.1.5 The Iterative Minimum Spanning Arborescence Algorithm in Figure 6.2 
returns a minimum spanning arborescence in polynomial time. 

The key step is to prove that the algorithm will terminate. In the following lemma 
we show that there is an arc a with either x a = 1 or x a = at each stage of the algorithm. 

Lemma 6.1.6 For any extreme point solution x to the directed LP, either there is an arc 
with x a = or there is an arc with x a = 1. 
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Before we begin the proof, let us recall that there exists a laminar family C such 
that it represents a linearly independent set of tight constraints (Corollary 6.1.4). The 
proof, by contradiction, is based on the token argument used in earlier proofs. The idea 
of the argument is to assume that there is no arc with x a = and x a = 1, and then 
derive a contradiction by showing that the number of constraints (that is, the number of 
sets in £.) is smaller than the number of non zero variables (that is, the number of arcs) - 
contradicting the Rank Lemma (that is, Corollary 6.1.4). 

As in the undirected case, we present three different counting arguments. While the 
first argument is the simplest for the minimum arborescence problem, the second argument 
can be generalized to the minimum cost rooted fc-connected subgraph problem, and the 
third proof can be generalized to the degree-bounded version of the minimum arborescence 
problem. 

Proof 1 (Global Counting Argument). Suppose for sake of contradiction < x a < 1. 
As we have x(S m (v)) > 1 for each v £ V \ {r}, hence we must have |<5 m (u)| > 2 for each 
v G V \ {r}. Thus 

W = £|**»l> £ 2 = 2|V|-2. 

vev vev\{r} 

On the other hand, from Lemma 6.1.4, we have the maximal linearly independent con- 
straints form a laminar family over the ground set V \ {r}. Prom Corollary 4.1.8 we 
have that \C\ < 2(\V\ — 1) — 1 = 2\V\ — 3, but this contradicts Corollary 6.1.4 since 
\A\ > 2\V\ - 2 > \C\. 

Proof 2 (Local Token Counting Argument). For each arc, one token is assigned 
to its head. So the total number of tokens assigned is exactly \A\. These tokens will be 
redistributed such that each subset S G C is assigned one token, and there are still some 
tokens left. This will imply \A\ > \C\ and thus contradicts Corollary 6.1.4. The following 
lemma shows that the redistribution is possible by an inductive argument on the forest L 
corresponding to the laminar family C. 

Lemma 6.1.7 For any rooted subtree of the forest L ^ with root S, the tokens assigned 
to vertices inside S can be distributed such that every node in the subtree gets at least one 
token and the root S gets at least two tokens. 

Proof The proof is by induction on the height of the subtree. The base case is when S is 
a leaf. Since x(5 m (S)) = 1 and there is no arc with x a = 1, there are at least two arcs in 
5 tn (S), and so S gets at least two tokens. 

For the induction step, let S be the root and R\,...,Rk be its children. By the 
induction hypothesis, each node in the subtree rooted at Ri gets at least one token and Ri 
gets at least two tokens. Since Ri only needs to keep one token, it can give one token to S. 
Suppose k > 2, then S can collect two tokens by taking one token from each of its children, 
as required. So suppose k = 1. If there is an arc a which enters S but not R\, then S can 
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collect two tokens by taking one token from R\ and one token from the head of a. Suppose 
such an arc does not exist, then 5 m {S) C 5 m (R). Since x(5 m (S)) = x(5 m (R)) = 1 and 
there is no arc with x a = 0, this implies 5 in (S) = 5 in (R). Hence x(5 in (S")) = x(<f n (.R)), 
but this contradicts the linear independence of the characteristic vectors for sets in £ (recall 
that £ can be chosen to satisfy the properties in Corollary 6.1.4). Therefore, such an arc 
must exist, and S can collect two tokens, as required. This completes the proof of the 
induction step. □ 

Prom Lemma 6.1.7, the number of tokens is at least \C\ + 1 which implies that 
\A\ > \C\, contradicting Corollary 6.1.4. 

Proof 3 (Local Fractional Token Counting Argument). The third counting argu- 
ment also starts from a laminar family C which satisfies the properties in Corollary 6.1.4. 
Instead of assigning tokens integrally, we assign tokens fractionally based on the following 
two rules. 

(i) For every arc a, we assign x a token to the smallest set in C containing its head. 

(ii) For every arc a, we assign 1 — x a token to the smallest set in C containing its tail. 

Thus the total number of tokens assigned is exactly \A\. To derive a contradiction, we 
show that each subset S G C has been assigned at least one token, and there are still some 
tokens left. This will imply \A\ > \C\ and thus contradicts Corollary 6.1.4. 

Lemma 6.1.8 Let S be any set in the C. Then S receives at least one token by the above 
assignment. 

Proof Let S be any set in C and let R±, . . . , R^ be the children of S where k > (k = 
implies S is a leaf). We have x(S m (S)) = 1, and for each Ri we have x{5 m {Ri)) = 1. 
Subtracting, we obtain x(5 in (S)) - ££ =1 x(<5 in (i?i)) = 1 - k. Let A s = 5 in (S) \ (U#™(i?;)) 
and Ar = (Lli5 m (Ri)) \ 5 m (S). Then the above equation can be rewritten as x(A$) — 
x(Ar) = 1 — k. Observe that the tokens received by S are exactly x a tokens for each 
arc a G As and exactly 1 — x a tokens for each arc in Ar. Hence, S receives exactly 
x(As) + |^4_r| — x(Ar) tokens, which is an integer since a; (As) — %{Ar) = 1 — k is an 
integer. If the tokens received by S is zero, then we must have As = and Ar = and 
then we have that x($ in (S)) = Yli x($ m (Ri)), contradicting the linear independence of 
constraints. Thus S receives at least one token. □ 

Therefore each set is assigned one token. To complete the argument, we need to show 
that there are still some extra tokens. This follows from observing that for any arc incident 
at the root, the 1 — x a > token is still unassigned as there is no set in C containing its 
tail. This completes the counting argument. 
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6.2 Minimum Cost Rooted fc-Connected Subgraphs 

It is easy to see that the problem of finding a rooted fc-arc-connected subgraph of minimum 
cost (requiring k arc-disjoint paths from r to every other vertex), generalizes the minimum 
arborescence problem. In fact, the LP for this problem is very similar to the directed LP 
for the arborescence (all the constraints require at least k instead of 1 on the right hand 
side). The separation oracle, the iterative algorithm and the counting argument are all 
similar for this problem as well. We leave the details to the exercises. 

We therefore shift our focus to the vertex connectivity problem, giving an LP for- 
mulation in the following. 



6.2.1 Linear Programming Relaxation 

A natural formulation of the minimum rooted fc-connected subgraph problem is to require 
that there are k internally vertex-disjoint paths from the root to every other vertex. As 
usual, an equivalent formulation as a "cut covering" problem is considered. For vertex- 
connectivity, it is not enough to just have k arcs entering a set, and a more sophisticated 
notion, bi-set, is needed. See Figure 6.3 for an example. 

Definition 6.2.1 A bi-set S = (So, Si) is a pair of subsets So, Si of V for which C 
Si C So C V. So is the outer member and Si is the inner member. An arc a = (u, v) is 
said to enter S = (So, Si) if u € V — So and v £ Si, which is denoted by x a £ S m (S). 




Fig. 6.3. In this example the bi-set S = (So, Si) is shown and the arc (u, v) enters S. 

Let us intuitively explain the role of bi-sets for this problem. A graph is rooted 
^-connected if upon removal of I < k vertices, there are still at least k — I vertex disjoint 
paths from r to other vertices. This is precisely the reason we count arcs coming in from 
V — So to Si - corresponding to the case where the vertices in So — Si are deleted. 
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Let 1Z be the set of all bi-sets S for which So Q V — r. We have the following bi-set 
LP formulation for the minimum rooted fc-connected subgraph problem. Note that, by 
Menger's theorem, a 0-1 solution to the bi-set LP corresponds to a rooted fc-connected 
subgraph; see Exercise 2.3 in Chapter 2. 



k — (\So\ — \Si\) V bi-sets Sell 
Vaei 

We can use a minimum-cut algorithm to design a separation oracle for the bi-set LP. 
Given a solution x for this problem, we can construct an auxiliary graph D' by expanding 
each vertex to an arc of capacity 1. All other arc capacities are the values of x a . It is easy 
to see that the solution x is a feasible solution to the bi-set LP if and only if the minimum 
cut separating the root from a vertex of V — r in D' has outgoing capacity at least k. 

It is tempting to try to extend the solution technique motivated by the arborescence 
case for this problem. This actually works to some extent, when we actually complete 
an iteration in the iterative algorithm, we pick an arc if its value was 1. However, this 
changes some of the constraints of the LP (the residual problem is no longer an instance of 
the rooted k vertex-connectivity problem), and therefore even the separation oracle could 
change in theory (in reality, all that is needed is a small addition to the above mentioned 
oracle). To apply the iterative method, we generalize the bi-set LP to the following. 



minimize c a x a 

aeA 

subject to x(5 in (S)) > 

Xa > 



minimize 



^ ^ Ca X a 

aeA 

subject to x(5 in (S)) > g(S) V bi-sets S e K 

> V ; ,1 



x n 



Initially g(S) is k — (\So\ — \Si\). The function g is required to satisfy a specific 
supermodularity property which we shall soon define. We are now in a position to explain 
the iterative algorithm for the rooted /c-connected subgraph problem. 



6.2.2 Iterative Algorithm 

The iterative algorithm to find an optimal rooted A;-connected subgraph is given in Fig- 
ure 6.4. 

Note that at each iteration, the separation oracle is exactly the one mentioned earlier 
where as a pre-processing step, we expand every vertex to an arc with x value of 1 in the 
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Iterative Rooted fc-Connected Subgraph Algorithm 

(i) Initialization F <— 0, g' <— g. 

(ii) While g' ^ do 

(a) Find an optimal extreme point solution x to the bi-set LP with the re- 
quirement function g' . Remove every arc a with x a = from D. 

(b) Find an arc a = uv with x a = 1, and update F <— FU{a}, D <— D\ {uv}, 
and set g'(S) <- m&x{g(S) - df(S),0}. 

(iii) Return F. 



Fig. 6.4. Rooted fc-Connected Subgraph Algorithm 

graph and also add in the arcs set to 1 so far by the algorithm, and look for a cut of value 
k as before. Details are left to the exercises. 



6.2.3 Characterization of Extreme Point Solutions 

The uncrossing technique can also be applied to bi-sets. For a bi-set S = (So, Si), the 
corresponding inequality x(5 m (S)) > g(S) defines the characteristic vector x(8 m (S)) in 
M.'" 4 ' : the vector has an 1 corresponding to each arc a G 5 m (S), and otherwise. Let 
T = {S | x(S in (S)) = g(S)} be the family of tight inequalities for an extreme point 
solution x to the bi-set LP. The following lemma shows that this family is closed under 
intersection and union. Before stating the lemma, we define bi-set intersections and unions 
(see Figure 6.5). 

Definition 6.2.2 Given two bi-sets S,T, define SdT := (So HTo, Si CiTj) and SUT : = 
(So U To, Si U Tj). Two bi-sets S, T are intersecting if Si D Tj, Si — Tj and Tj — Si are 
nonempty, or Si D Tj, So H To, So — To and Tq — So are nonempty. 




Fig. 6.5. In this example the bi-set S = (So, Si) and bi-set T = (T ,Tf) intersect. 
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Lemma 6.2.3 If bi-sets S,T £ J- and S and T are intersecting, then both S(~)T and SUT 
are in T. Furthermore, x($ in (S)) + x(5 in (T)) = x($ in (S D T)) + x{S in (S U T)). 



Proof 

g(S) + g(T) = x(5 m (S))+x(5 m (T)) 

> x(5 in (SriT)) + x(5 in (SuT)) 

> g(SnT) + g(SUT) 

> g(S) + g(T) 

The equality is by the fact that S, T € T . The first inequality follows from the sub- 
modularity of the bi-set function d m (S), which can be shown in a manner similar to the 
corresponding submodularity of the set function d in (S) in the arborescence problem (see 
the exercises). The second inequality follows from the constraints in the bi-set LP. The 
last inequality is because of the property we require the function g to satisfy - intersecting 
bi-supermodularity. We will prove in Proposition 6.2.7 that the function g' is intersecting 
bi-supermodular at each stage of the iterative algorithm. 

Definition 6.2.4 A function g is intersecting bi-supermodular if for all intersecting bi-sets 
S and T, g(S) + g(T) < g(S U T) + g(S n T). 

Therefore, equality holds everywhere, and thus x(S in (Sr\T)) = g(SC\T) and x(5 tn (SU 
T)) = g(SUT), which implies that S(lT and SUT are also in T. Moreover, since x a > 
for every arc a, we have x(^ n (^)) + x{S in (T)) = x(<f n (S n T)) + x($ in (S U T)). □ 

Denote by span(F) the vector space generated by the set of characteristic vectors 
{x($ m (S)) I S G J 7 }- A family of bi-sets is bi-laminar if no two bi-sets are intersecting. In a 
bi-laminar family, the inner members form a laminar family, and if Xj C Yj then Xq C Yq 
or Yo C Xo (see the exercises) . The following lemma says that an extreme point solution is 
characterized by tight inequalities whose corresponding sets form a bi-laminar family. The 
proof is similar to the corresponding proof for the spanning tree problem (see Lemma 4.1.5) 
and is omitted here. 

Lemma 6.2.5 If C is a maximal bi-laminar subfamily of T , then span(C) = span(J-). 
Lemma 6.2.5 and the Rank Lemma imply the following. 

Corollary 6.2.6 Let x be any extreme point solution to the bi-set LP. Then there exists a 
bi-laminar family C such that 

(i) x(5 in (S)) = g(S) for all S € C. 

(ii) The vectors in {x(^ m (5')) : S G £} are linearly independent. 
(hi) \A\ = \C\. 
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6.2.4 Correctness and Optimality 



Firstly, for the inductive procedure to work, we want g to be bi-supermodular initially. 
Recall that we initialized g(S) as k — (\So\ — \ It is not too difficult to verify that this 
g is indeed bi-supermodular. 

Proposition 6.2.7 The function g' is an intersecting bi-supermodular function. 

Proof Let S and R be two intersecting bi-sets. 

g(S) + g(R) = (k-lSol + lS^ + ik-lRol + lRil) 

= (k - \s n Ro\ + \Si n i? 7 |) + (k - \S u R \ + IS/ u Ri\) 
= g{S n R) + g(S U R) 

Since g is an intersecting bi-supermodular function, and g' is obtained from g by subtracting 
an intersecting bi-submodular function d l p{S) (the subset of arcs set to 1 in each iteration), 
g' is also an intersecting bi-supermodular function. □ 

Assuming the algorithm terminates successfully, the returned solution is a minimum 
rooted fc-connected subgraph - the proof of optimality is by now a standard induction 
argument which we omit here. 

Theorem 6.2.8 The Iterative Rooted k-Connected Subgraph Algorithm in Figure 6.4 re- 
turns a rooted k-connected subgraph of minimum cost in polynomial time. 

The key to prove the correctness is to show that the algorithm will terminate. This 
is very similar to a counting argument we employed for minimum arborescences. 

Lemma 6.2.9 For any extreme point solution x to the bi-set LP, either there is an arc 
with x a = or there is an arc with x a = 1. 

Suppose for a contradiction that there is no arc with x a = or x a = 1. The proof 
starts from a bi-laminar family £ which satisfies the properties of Corollary 6.2.6. A bi- 
laminar family £ also naturally defines a forest L as follows: Each node of L corresponds 
to a bi-set in £, and there is an edge from bi-set R to bi-set S if R is the smallest set 
containing S, where a bi-set R contains a bi-set S if Si C Rj and So Q Ro- The goal 
of the counting argument is the same - assuming there is no arc with x a = and x a = 1, 
derive a contradiction by showing that the number of inequalities (that is, the number of 
sets in C) is smaller than the number of variables (that is, the number of arcs). 

For each arc, one token is assigned to its head. So the total number of tokens assigned 
is exactly |^4|. To derive a contradiction, these tokens will be redistributed such that each 
bi-set S G C is assigned one token, and there are still some excess tokens left. This will 
imply \A\ > \C\ and thus contradict Corollary 6.2.6. The following lemma shows that such 
a redistribution is possible. 
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Lemma 6.2.10 For any rooted subtree of the forest L 7^ with root S, the tokens assigned 
to vertices inside Si can be distributed such that every node in the subtree gets at least one 
token and the root S gets at least g(S) + 1 tokens. 

Proof The proof is by induction on the height of the subtree. The base case is when S is 
a leaf. Since x(5 m (S)) = g(S) and there is no arc with x a = 1, there are at least g(S) + 1 
arcs in S m (S), and so S gets at least g(S) + 1 tokens. 

For the induction step, let S be the root and Ri,...,Rk be its children. By the 
induction hypothesis, each node in the subtree rooted at Ri gets at least one token and 
Ri gets at least g(Ri) + 1 tokens. Since Ri only needs to keep one token, it can give g{Ri) 
tokens to S. Let g(R) = Yli=i9(Ri)- Three cases are considered. 

(i) g(S) < g(R)- Then S can collect g(R) > g(S) + 1 tokens from its children. 

(ii) g(S) > g(R). Since there is no arc with x a = 1, there must be at least g(S)—g(R) + l 
arcs entering S but not its children. So S can take g(R) tokens from its children 
and g(S) — g(R) + 1 tokens from the heads of those arcs. Therefore, S can collect 
at least g(S) + 1 tokens, as required. 

(iii) g(S) = g(R). If there is an arc a which enters S but not any of its children, then 
S can collect g(S) + 1 tokens by taking g(S) = g(R) tokens from its children and 
at least one more token from the head of such arc, as required. Suppose such an 
arc does not exist, then 5 in (S) C U^ =1 5 in (Ri). Since x{5 in (S)) = J2i =1 x(S in (Ri)) 
and there is no arc with x a = 0, this implies that 5 in (S) = L)^ =1 5 m (Ri). Hence 
x(o~ m (S)) = Yli=i x(d m (Ri)), but this contradicts the linear independence of the 
characteristic vectors for bi-sets in C Therefore such an arc must exist, and thus S 
can collect g(S) + 1 tokens, as required. 

This completes the proof of the induction step. □ 

From Lemma 6.2.10, the number of tokens is at least \C\ + 1 which implies that 
\A\ > \C\, contradicting Corollary 6.2.6. This completes the proof of Lemma 6.2.9, and 
hence Theorem 6.2.8 follows. 



6.3 Minimum Bounded Degree Arborescence 

The minimum bounded degree arborescence problem (MBDA) is denned as follows: Given 
a directed graph G = (V, A), a cost function c : A — > R and an out-degree upper bound B v 
for each vertex v G V, the task is to find an arborescence of minimum cost which satisfies 
all the out-degree bounds. 

First we present a (2,2B V + 2)-approximation algorithm for the minimum bounded 
degree arborescence problem. By this, we mean that the algorithm outputs an arborescence 
whose cost is twice that of the LP relaxation for the original problem and whose out-degree 
at any node v is at most 2B V + 2. Thus it is a pseudo-approximation or bicriteria approx- 
imation algorithm that both violates the degree bounds as well as delivers a suboptimal 
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cost solution. In the next section, we will present an algorithm with only an additive error 
of 2 on the degree bounds but when there are no costs on arcs (the unweighted case) . 

6.3.1 Linear Programming Relaxation 

The following is a natural extension of the directed LP for this degree-bounded problem 
where the degree bounds are imposed only on a subset W of the vertices. 

minimize c a x a 

a<=A 

subject to x(5 in (S)) > 1 V S C V - r 

x{5 out {v)) < B v \f veW 

x a > V a G A 

This linear program can be solved by using a minimum cut algorithm as a separation 
oracle as in the case of the directed LP for minimum cost arborescences. 

To make the formalism sufficiently general to handle residual problems arising in 
the iterative procedure, we use an LP for a generalization of arborescences to more general 
cut-functions / : S — > Z + . Recall that a pair of sets A, B C V are intersecting if AC\B / 0, 
4-5^0 and B — A / 0. A function / is intersecting supermodular if for every pair of 
intersecting sets A, B C V we have 

f(A) + f(B)<f(AuB) + f(AnB). 

In the following f(S) is a 0-1 intersecting supermodular function. By setting f(S) = 
1 for all subsets S that do not contain the root r (this is intersecting supermodular, see ex- 
ercise 2.12), we get an LP relaxation, denoted by LP m bda(G, f,B, W), for the arborescence 
problem below. Here B denotes the vector of out-degree bounds on the subset of vertices 
W. 

minimize ^ 

a&A 

subject to x(5 in (S)) > f(S) V S C V - r 
x{5 out {v)) < B v V v G W 

x a > V a £ A 

To find a separation oracle for this problem, we need to check whether x(S' in (S)) — 
f(S) > for each S C V — r. For general supermodular function /, we can use an efficient 
algorithm for submodular function minimization to check this, since x(5 tn (S)) — f(S) is 
a submodular function. For the function / arising from the minimum bounded degree 
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arborescence problem, we can use an efficient minimum cut algorithm for this task, as we 
did for the minimum rooted A;-connected subgraph problem. 

6.3.2 Characterization of Extreme Point Solutions 

The proof of the following lemma follows from standard uncrossing arguments as in the 
minimum arborescence problem. 

Lemma 6.3.1 For any extreme point solution x to LP m bd a , there exist a set T C W and 
a laminar family C such that 

(i) x{8 out {v)) = B v for each v £ T and x(S in (S)) = 1 for each S £ C. 

(ii) The vectors in {x(^° ut ( v )) '■ v £ T} U {x(5 m (S)) : S £ C} are linearly independent. 
(hi) \A\ = \C\ + \T\. ' 



6.3.3 Iterative Algorithm 

The iterative algorithm is similar to that for the minimum bounded degree spanning tree 
problem in Chapter 4, except that it has a relaxation step as well as a rounding step, in 
which we choose an arc a even though x a < 1. 

Iterative Minimum Bounded Degree Arborescence Algorithm 

(i) Initialization F «- 0, /' = /. 

(ii) While /' do 

(a) Compute an optimal extreme point solution x to LP m bda(G, f, B, W). Re- 
move every arc a with x a = 0. 

(b) (Rounding): If there is an arc a = (u,v) with x a > ^, then update 
F<-FU {a} and set B u ^B u -\. 

(c) (Relaxation): If there exists a vertex v £ W with d out {v) < B v + 3, then 
set W <r- W - v . 

(d) Update f'(S) <- max{/(5) - df(S),0}. 

(iii) Return F. 



Fig. 6.6. Minimum Bounded Degree Arborescence Algorithm 



6.3.4 Correctness and Performance Guarantee 

We first prove that the algorithm has the claimed performance guarantee, assuming that 
it terminates successfully. 
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Theorem 6.3.2 The iterative algorithm in Figure 6.6 is a (2,2B V + 2)- approximation 
algorithm for the minimum bounded degree arborescence problem. 

Proof First we prove that the cost of the arborescence returned by the algorithm is at 
most twice the cost of the initial LP solution. The proof is by induction on the number of 
iterations executed by the algorithm. For the base case that requires only one iteration, 
the theorem follows since it rounds up a single arc a with x a > \- For the induction step, 
let a' be the arc with x a > > \ in the current iteration. Let /' be the residual requirement 
function after this iteration and let F' be the set of arcs picked in subsequent iterations 
for satisfying /'. The key observation is that the current solution x restricted to A — a' 
is a feasible solution for satisfying /' (by checking the connectivity constraints and also 
the degree constraints), and thus by the induction hypothesis, the cost of F' is at most 
2 J2 a <=A-a' °a x a- Consider F := F' U a' which satisfies / (by the definition of /'). The cost 
of F is: 

cost(F) = cost(F') + c a > < 2 ^2 c a x a + c a > < 2 ^ c a x a , 

a£A~a' aeA 

where the last inequality follows because x a > > \. This implies that the cost of F is at 
most twice the cost of an optimal fractional solution. 

Next we show that the degree of any vertex v is at most 2B V + 2. At any iteration, 
let F denote the set of edges selected and let B' v denote the current residual degree bound 
of v. While the degree constraint of v is present, <If{v) = 2(B V — B' v ). This is because, 
at any iteration, whenever we include an edge e G S(v) in F, we reduce B' v by half and 
hence the equality holds true. When the degree bound for the vertex v is removed then 
less than B' v + 3 edges are incident at v. In the worst case, we may select all these edges 
in the solution. Hence, 

d F (v) < 2(B V - B' v ) + B' v + 3 < 2B V + 3. 
Since dp(v) and B v are integers, this implies that dp(v) < 2B V + 2, as required. □ 

We now focus on using a counting argument to prove the following lemma which 
guarantees that the algorithm terminates successfully. 

Lemma 6.3.3 An extreme point solution x to LP m i,da must satisfy one of the following. 

(i) There is an arc a with x a > |. 

(ii) There is a vertex v with d on *(v) < B v + 3. 

Proof The proof uses the local fractional token argument. Each arc is assigned two tokens, 
for a total of 2\A\ tokens. For each arc a, 1 — x a token is assigned to its tail, and l + x a token 
is assigned to its head. We shall show that if none of the above conditions are satisfied, 
then each set in £ and each degree constraint in T can collect two tokens, and there are 
some tokens left. This would imply \A\ > \C\ + |T|, which contradicts Lemma 6.3.1. 
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For each tight vertex v in T, it collects 



£ (l-x a ) = d° ut (v)- £ 




(«) - 5« > 3 




tokens; the second equality follows because t> is tight, and the last inequality follows because 
the condition in the lemma is not satisfied. This shows that each degree constraint in T 
can collect at least three tokens. Hence each degree constraint has at least one extra token. 
In the following, each node with a degree constraint will contribute its extra token to the 
smallest set in the laminar family containing it. 
For a leaf node S G C, it collects 



tokens; the second equality follows because S is tight and so x(5 in (S)) = 1, and the last 
inequality follows because there is no arcs with value at least 1/2 and hence d in (S) > 3. 
This shows that each leaf node in C can collect at least four tokens. Hence each leaf node 
has at least two extra tokens to start us off in the bottom-up induction over the laminar 
family. 

We argue by induction going bottom up in the laminar family C that we can assign 
the tokens in the subtree of S so that every set gets at least two tokens and the root gets 
two extra tokens. We proved the base case for leaves in the previous paragraph. Consider 
a non-leaf node S £ C, and let its children be i?i, . . . , R\. If I > 2, then S can collect two 
extra tokens from each child by the induction hypothesis, and hence S has at least two 
extra tokens, as required. So assume S has only one child R\, and S can collect two tokens 
from R\ and needs two more tokens. Since x(8 m (S)) = x(<5 m (i?i)) = 1 and x(£ m (5)) / 
X (S in (Ri)), there are arcs p G P := 5 in (S) -5 in {R\) and q G Q : = (5 iri (i?i) - 5 in (S). Every 
arc p £ P can contribute 1 + x p token to S. If the tail of q G Q is in T (the set of vertices 
with tight degree constraint), then its tail contributes 1 token to S since it has a degree 
constraint and by the redistribution above, every such node gives its one extra token to 
the smallest set in the laminar family containing it. If the tail of q is not in T, then q can 
contribute 1 — x q token to S. In this case, note that since x{5 m {S)) = x(5 m (R\)) = 1, we 
have ^2 p£ pX p = J2 q eQ x i an( ^ ^ nus & can conec t two more tokens from arcs in P and Q, 
and this completes the induction step. There are always some tokens left at the root node 
in £, and this completes the proof. □ 



In this section we present an algorithm for the unweighted problem (without arc costs) that 
outputs an arborescence that violates the out-degrees by at most two. However, as opposed 
to the minimum bounded degree spanning tree problem, to achieve an additive guarantee 
on the degrees, the cost of the solution is no longer bounded. In fact, this tradeoff is shown 
to be unavoidable using this linear programming relaxation for the problem [9]. 



£ (1 + = ^(5)+ £ 



X a = d" 



(5) + 1 > 4 




6.4 Additive Performance Guarantee 



104 



6-4-1 Iterative Algorithm 



The iterative algorithm is similar to the additive one approximation for the minimum 
bounded degree spanning tree problem. The algorithm removes degree constraint one by 
one, and include all arcs incident on these vertices as candidates for the final solution, and 
remove them from the graph. In the end a subset of arcs that make up an arborescence 
among the candidates is output. The algorithm is presented in Figure 6.7. 

Iterative Additive Arborescence Algorithm 

(i) Initialization F 4- 0, /' = /. 

(ii) While A \ F / do 

(a) Find an optimal extreme point solution x to LP m bd a (G, f, £>, W). Remove 
every arc a with x a = 0. 

(b) (Relaxation): If there exists a vertex v 6 W with d out {v) < B v + 2, then 
set W 4- W - v and update F <- F U {5 out (v)}. 

(c) Update f'(S) <- max{f(S) - df(S),0} and A <- A - F. 

(iii) Return any arborescence in F. 

Fig. 6.7. Additive Arborescence Algorithm 

The degree constraint is violated only in Step 2(c) by at most two. So if the algorithm 
terminates successfully, then the algorithm is an additive two approximation algorithm for 
the unweighted bounded degree arborescence problem. 

6-4-2 Correctness and Performance Guarantee 

The following lemma shows that the algorithm always terminates successfully. The proof 
uses the characterization of the extreme point solutions of LP m bd a - 

Lemma 6.4.1 In any extreme point solution x to LP m bd a , there is a vertex v G W with 
out-degree at most B v + 2. 

Proof The proof uses a local fractional token argument. Each arc is assigned one token, 
for a total of \A\ tokens. For each arc a, 1 — x a token is assigned to its tail, and x a token is 
assigned to its head. We shall show that if the above condition is not satisfied, then each 
set in £ and each degree constraint in T can collect one token, and there are some tokens 
left. This would imply \A\ > \C\ + |T|, contradicting Lemma 6.3.1. 

Each vertex v with nonzero out-degree must be in W. Hence it collects 

(1 - x a ) = d out {v) - x{5 ou \v)) > d ou \v) -B v >3 

a£5 out (v) 

tokens; the first inequality follows because of the out-degree constraint at v, and the last 
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inequality follows because the condition in the lemma is not satisfied. This shows that 
each vertex with non-zero out-degree has at least two extra tokens. Each such vertex with 
an out-degree constraint distributes its two extra tokens to the smallest set in C containing 
it. 

A leaf node S G C collects x(5 tn (v )) = 1 token. Furthermore, S has at least one extra 
token if d out (S) > 1 (actually at least two extra tokens but we only need one extra token 
for the induction to work). We call S with d out (S) = a sink node. Note that when S is 
a leaf node, it gets at least one token if it is a sink node and at least two if it is a non-sink 
node. We prove inductively (going bottom-up in C) that we can redistribute the tokens in 
the subtree of any non-leaf node S G C so that every set gets at least one token and the 
root S gets at least two tokens. For the induction step, consider a non-leaf node S G C, 
and let its children be Ri, . . . , R[. If S has at least two children that are non-leaf nodes or 
are leaf nodes that are non-sink nodes, then S can collect one extra token from each such 
child by the induction hypothesis, and hence S has at least two tokens, as required. So 
assume S has at most one child R\ that is a non-leaf node or a leaf node that is a non-sink 
node. 



Fig. 6.8. In this figure, set S has children R\, . . . , Ri where all except R\ are leaf as well sink nodes. 
Thus the tail u of arc a € <5 m (i?i) \ S m (S) must have 2 extra tokens which it gives to S. 

Since x{5 in (S)) = x(5 in (R 1 )) = 1 and x(^ n (<S)) / x(^ n (^i)), there is an arc 
a G 5 tn (R\) — 5 tn (S). Since i?2, . . . ,Ri are sink nodes, the tail of a is not contained in 
Ri U i?2 U . . . U Ri but is in S. Furthermore, since the tail of a has positive out-degree in 
the support, it has two extra tokens by the argument in the previous paragraph that it 
assigns to S, as required. The roots of C have extra tokens left, completing the proof. □ 



The directed LP formulation for the minimum spanning arborescence problem is due to 
Edmonds [46]. The first polynomial-time algorithm for the minimum rooted fc-connected 
subgraph problem was given by Frank and Tardos [58] via the use of submodular flows 



S 




Notes 
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(which we will introduce in Chapter 7). Recently Prank [57] showed that the minimum 
rooted /c-connected subgraph problem can actually be reduced to the matroid intersection 
problem, and that the bi-set LP for this problem is totally dual integral (TDI) . The bicri- 
teria approximation algorithm for the minimum bounded degree arborescence problem is 
by Lau, Naor, Salavatipour and Singh [110], while the additive approximation algorithm 
for the unweighted problem is due to Bansal, Khandekar and Nagarajan [9]. 

Exercises 

6.1 Show the equivalence of the directed LP and the flow-based LP in Section 6.1.1. 

6.2 Write a compact formulation for the spanning tree problem. (Hint: Use the com- 
pact formulation for the arborescence problem.) 

6.3 Show the application of the iterative method to formulate an integral LP for the 
rooted /c-arc-connected subgraph problem. Use the same outline we have used 
as in the earlier chapters: LP formulation, properties of extreme point solutions, 
iterative algorithm and its correctness and optimality. 

6.4 Work out the details of the separation oracle for the bi-set LP relaxation for this 
problem. Also, show how the separation oracle can be adapted for the LP formu- 
lations that arise when some of the arcs are already chosen in a current solution. 

6.5 A function g on bi-sets is intersecting bi-submodular if g(S) + g(T) > g(S D T) + 
g(S U T) holds for any two intersecting bi-sets S and T. Prove that the bi-set 
function d in (S) is intersecting bi-submodular. 

6.6 Show that in a bi-laminar family, the inner members form a laminar family, and if 
Xi C Yi then X C Y or Y C X . 

6.7 Prove Lemma 6.2.5. 

6.8 Can the approximation algorithms for the minimum bounded degree arborescence 
problem be extended to the minimum bounded degree rooted A:-edge-connected 
subgraph problem? 
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7 

Submodular Flows and Applications 



t 

Quoting Lovasz from his paper "Submodular Functions and Convexity" [116]: 



"Several recent combinatorial studies involving submodularity fit into the following pattern. Take 
a classical graph-theoretical result (e.g. the Marriage Theorem, the Max-flow-min-cut Theorem 
etc.), and replace certain linear functions occurring in the problem (either in the objective function 
or in the constraints) by submodular functions. Often the generalizations of the original theorems 
obtained this way remain valid; sometimes even the proofs carry over. What is important here 
to realize is that these generalizations are by no means l'art pour l'art. In fact, the range of 
applicability of certain methods can be extended tremendously by this trick." 



The submodular flow model is an excellent example to illustrate this point. In this chap- 
ter, we introduce the submodular flow problem as a generalization of the minimum cost 
circulation problem. We then show the integrality of its LP relaxation and its dual using 
the iterative method. We then discuss many applications of the main result. We also show 
an application of the iterative method to an NP-hard degree bounded generalization and 
show some applications of this result as well. 

The crux of the integrality of the submodular flow formulations will be the property 
that a maximal tight set of constraints form a cross-free family. This representation al- 
lows an inductive token counting argument to show an 1-element in an optimal extreme 
point solution. We will see that this representation is precisely the one we will eventually 
encounter in Chapter 8 on network matrices. 



t © Copyright 2011 by Lap Chi Lau, R. Ravi and Mohit Singh. Published 2011 by Cambridge 
University Press. 
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7.1 The Model and the Main Result 



7.1.1 Minimum- Cost Flows and Minimum- Cost Circulations 

Given a directed graph D = (V, A), a source vertex s G V, a sink vertex t G V, a capacity 
function c : A — > Q+, a cosi function w : A — > Q, and a value A;, the minimum cost s-t flow 
problem is to find an s-t flow of value k with x a < c a that minimizes cost(x) := Yla&A w a' x a- 

Recall that a flow function x assigns non-negative flow values to each arc that obeys 
flow conservation constraints at all nodes except the source s and the sink t. The flow 
conservation insists that the flow coming via the in-arcs equals that going via the out-arcs. 
With these constraints at all nodes except s and t, the net out-flow from s equals the net 
in-flow into t which is the value of this s — t flow. The capacity constraints mentioned 
above insist that the flow routes through an arc not exceed its capacity to carry flow. 

Related to this classical problem is the minimum cost circulation problem, where 
each arc a has a lower bound d a and an upper bound c a , and the goal is to find a minimum 
cost circulation (i.e., a flow function that is conserved at all nodes including s and t) that 
satisfies capacity lower and upper bounds: d a < x a < c a . 

The problem of finding a minimum-cost s-t flow of value k can be easily reduced 
to a minimum-cost circulation problem: just add an arc ciq = (t, s) with d ao = c ao = k, 
and w ao = 0. Also let d a = for each arc a / ao- Then a minimum cost circulation in 
the extended directed graph gives a minimum cost flow of value k in the original directed 
graph. 

Also the problem of finding a maximum value s-t flow can be easily reduced to a 
minimum-cost circulation problem in the extended directed graph: just define d ao = 0, 
c ao = oo, and w a() = — 1- Moreover, set w a = for each a ^ ao. Then a minimum cost 
circulation gives a maximum value s-t flow. 

Edmonds and Karp showed that the minimum cost circulation problem is solvable in 
polynomial time. One can now generalize this problem by extending the flow conservation 
constraints at each node in the circulation from singleton nodes to a collection C of node 
subsets. Also, the right hand side of these constraints which are all zero for the circulation 
problem can be generalized to be a submodular function f(S) for the constraint for set S. 
These two generalizations of the circulation problem give the submodular flow problem. 



7.1.2 Generalizing to Submodular Functions 

Let D = (V, A) be a directed graph and let C be a crossing family of subsets of V (that is, 
if T,U G C with TDU / 0, TUU^ V,T-U / and U -T ^ 0, then T P\U,T(JU G C). 
Recall that a function / : C — > M is called crossing submodular, if for all U,T G C with 
U n T / 0, U U T / V, U - T / and T - U / one has 

f(U) + f(T) > f(UnT) + f(UUT) 
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Given such D, C, f, a submodular flow is a function x € ~R A satisfying: 

x(5 in (U)) - x{5 out {U)) < f(U) for each U G C, 

where x(5' m (U)) is a shorthand for J2ae8 m (U) Xa an( ^ similarly for x(S out (U)). The set P 
of all submodular flows with the bound constraints d a < x a < c a is called the submodular 
flow polyhedron. 

Note that the minimum-cost circulation which we generalized from can easily seen 
to be a special case of a maximum-cost version of the submodular flow problem where the 
costs are -1 for every arc. Take C = {{v} \ v £ V}; trivially C is a crossing family. Set 
/ = 0; obviously / is a crossing submodular function. The maximum-cost submodular 
flow problem becomes: 

maximize w a x a 

subject to x(5 in {v)) - x{5 out {v)) < V?) C V 

da < X a < C a Vo G A 

Note that no inequality at a vertex can be a strict inequality, and so the solution must be 
a maximum cost circulation. We refer to the above constraint system as constituting the 
submodular flow polyhedron. 

The main result of this chapter is the following theorem. 



Theorem 7.1.1 If f is a crossing submodular function, then both the primal and the dual 
of the submodular flow polyhedron are integral. 

The result has myriad consequences many of which we will describe in Section 7.4. 



7.2 Primal Integrality 

In this section, the integrality of the submodular flow polyhedron will be derived via an 
iterative method. We derive the result for the special case when the bound functions are 
d a = and c a = 1 for all arcs a. The extension to more general integral bounds will be 
immediate and the subject of the exercises. We only use integral submodular function / 
in the sequel (but not necessarily nonnegative) . 
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7.2.1 Linear Programming Relaxation 



The linear programming formulation LP sm f(D,f,w) is restated as follows, 
maximize w a x a 

subject to x(5 in (S)) -x(5 OM *(5)) < f(S) V5C7 

< x a < 1 VaGi 

As you might have expected by now, this linear program can be solved by the ellipsoid 
method, using the polynomial time algorithm for minimizing submodular functions as a 
separation oracle. To see this, note that the function x(S in (S))—x(5 out (S)) is modular since 
this is exactly J2v&s( x ( din ( v )) ~ x(S out (v))). Thus the function g(S) = f(S)-(x{5 in {S))- 
x(S out (S))) is submodular and the separation problem is equivalent to checking whether 
the minimum of g(S) is negative or not. 



7.2.2 Characterization of Extreme Point Solutions 

As in previous problems, the uncrossing technique is used to find a "good" set of tight 
inequalities that defines an extreme point solution to the submodular flow LP. For a set 
SCV, the corresponding inequality x(S in (S)) — x(5 out (S)) < f(S) defines a characteristic 
vector x(S) in M) A \; the vector has value 1 corresponding to each incoming arc a G 5 in (S), 
value -1 corresponding to each outgoing arc a G S out (S), and otherwise. Let F = 
{S | x(S m (S)) — x(S out (S)) = f(S)} be the family of tight inequalities for an extreme point 
solution x in the submodular flow LP. The following lemma shows that this family is a 
crossing family. The proof is by now standard - you should work out the details to see 
how nicely the submodular constraint fits with our usual proof approach (e.g., 4.1.4 and 
5.2.2) for this lemma. 

Lemma 7.2.1 // S,T G F and S C\T / and S U T / V , then both S C\T and S U T are 
in F. Furthermore, x(S) + x(T) = x(S flT) + x{S U T). 

Denote by span(F) the vector space generated by the set of vectors {x(S) \ S G F}. 
Recall that two sets X, Y are crossing if XdY, X — Y, Y — X and V — (XUY) are nonempty. 
A family of sets is cross-free if no two sets are crossing. The following lemma says that 
an extreme point solution is characterized by tight inequalities whose corresponding sets 
form a cross-free family. The proof follows the same lines as in Lemma 4.1.5. 

Lemma 7.2.2 If C is a maximal cross-free subfamily of F, then span(C) = span(F). 

Lemma 7.2.2 and the Rank Lemma imply the following characterization of extreme 
point solutions. 
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Lemma 7.2.3 Let x be any extreme point solution to LP sm f(D, f, w) with < x a < 1 for 
each arc a £ A(D). Then there exists a cross-free family C such that 

(i) x(5 in (S)) - x(5 out (S)) = f(S) for each S e C. 

(ii) The vectors in {x(S) '■ S £ C} are linearly independent. 

(iii) \C\ = \A{D)\. 

7.2.3 Iterative Algorithm 

An iterative procedure is used to find a minimum cost submodular flow from an optimal 
extreme point solution to LP sm f(D,f,w). We fix the values of the arcs with x a = or 
x a = 1, and update the submodular function accordingly. The algorithm is described in 
Figure 7.1. 

Iterative Submodular Flow Algorithm 

(i) Initialization F = ®, f <— f. 

(ii) While A / do 

(a) Find an optimal extreme point solution x to LP sm f(D, f, w). Delete every 
arc a with x a = from D. 

(b) For each arc a = uv with x a = 1, delete a from L>, add a to F, and update 
f(S) <-f(S)- df{S) + c^ u *(S) for each SCV. 

(iii) Return the solution x. 



Fig. 7.1. Submodular Flow Algorithm 



7.2.4 Correctness and Optimality 

One nice feature of the submodular flow problem is self-reducibility: after an arc a = 
uv with x a = 1 is picked and the submodular function is modified appropriately, the 
residual problem is still a submodular flow problem. Assuming the algorithm terminates 
successfully, the returned solution is a maximum cost submodular flow using the by-now 
standard induction argument. 

Theorem 7.2.4 The Iterative Submodular Flow Algorithm in Figure 7.1 returns a mini- 
mum cost submodular flow in polynomial time. 

The following lemma shows that the algorithm will terminate. 

Lemma 7.2.5 For any extreme point solution x to LP sm f(D, f,w), either there is an arc 
with x a = or there is an arc with x a = 1. 
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The proof of Lemma 7.2.5 starts from a cross-free family C which satisfies the proper- 
ties of Lemma 7.2.3. Prom a cross-free family, a laminar family £ is constructed as follows. 
Let r £ V be an arbitrary vertex. 

C:={XCV-r : X€C}U{XCV-r : V - X € C} 

The first set includes the members of C which does not contain r, and so is laminar. The 
second set includes the members of C which contain r, and so their complements form a 
laminar family. Also it can be checked that a member in the first set does not intersect a 
member in the second set (otherwise they are crossing in C), and so £ is a laminar family. 
Note that since the constraints in C are linearly independent, there are no repeated sets in 
the laminar family C. 

As usual, a counting argument will be used to derive a contradiction - assuming 
there is no arc with x a = or x a = 1, then the number of inequalities is smaller than 
the number of variables. For each arc, one token is assigned to its head and one token is 
assigned to its tail. So the total number of tokens assigned is exactly 2\A\. These tokens 
will be redistributed such that each set S G C is assigned two tokens, and there are still 
some excess tokens left. This will imply \A\ > \C\ and yield a contradiction. Let L be 
the forest corresponding to the laminar family C The following lemma shows that the 
redistribution is possible. 

Lemma 7.2.6 For any rooted subtree of the forest L ^ with root S, the tokens assigned 
to vertices inside S can be distributed such that every node in the subtree (including the 
root S) gets at least two tokens. 

Proof The proof is by induction on the height of the subtree. The base case is when S 
is a leaf. Since x(5 in (S)) — x(5 out (S)) = f(S) and there is no arc with x a = 1 or x a = 0, 
either there is at least two incoming arcs, or there is at least two outgoing arcs, or there is 
at least one incoming and one outgoing arc. In any case, S can collect two tokens. 

For the induction step, let S be the root and R\,...,Rk be its children. By the 
induction hypothesis, each node in the subtree rooted at Ri gets at least two tokens and 
Ri gets at least two tokens. To prove the induction step, S needs to collect two more 
tokens. If there is an arc which enters S but not any i?,, or there is an arc which leaves S 
but not any then S can collect one token from this arc. Suppose, to the contrary, that 
such an arc does not exist. Then 

k 

sign(S)(<nS) " S^iS)) = J>gn(^)(<H^) " S out (R t )), 

i=i 

where sign(L r ) = 1 if the set U G C and sign({7) = — 1 if the set V — U £ C, since each arc 
with the tail in Ri and the head in Rj with i ^ j is canceled out in the summation. This 
implies that 

k 

sign(5)-x(5) = ^sign(i? i )-x(^), 
i=i 
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contradicting the linear independence of the characteristic vectors. Therefore, there must 
be at least an arc which has an endpoint belonging to S but not Ri, . . . , Rk- If a is the 
only such arc, then 

k 

x(a) = sign(S) • (x(5 m (S)) - x(5 out (S))) - ^sign(i^) • (x(5 m (Ri)) - x(5 out (Ri))) 

i=i 

k 

= siga(S)-f(S)-^2siga(R i )-f(R i ). 

i=i 

Since / is an integer-valued function, then x(a) must be an integer, a contradiction. So 
there must be at least two such arcs, and therefore S can collect two extra tokens, as 
required. □ 

It remains to show that there is some unused tokens. Consider the roots Si, . . . , Si 
in the laminar family C. If there is an arc with one endpoint not in any Si, then the 
token in that endpoint is unused, as required. Otherwise, if no such arc exists, then the 
characteristic vectors of Si are linearly dependent since Yl\=i s ^S n {'Si)(S m (Si) —5 out (Si)) = 
0, a contradiction. This completes the proof of Lemma 7.2.5. 



7.3 Dual Integrality 

Consider the dual linear program for the submodular flow problem. We first write the dual 
linear program of the submodular flow linear program without the upper bound constraints 
on the arcs. The argument can be simply extended to deal with upper bounds on arcs, 
and is left as an exercise. Denote the LP below by LPd sm f(D,f,w). 

minimize ^ f(S) y(S) 

scv 

subject to y(S) ~ y(^) — w a VaCi 

S:aeS in (S) S:aeS° M (S) 

y{S) > VS 1 c V 

The uncrossing technique can be used to prove the following claim, whose proof is 
similar to that of Claim 5.4.1 and Claim 5.4.3. 

Claim 7.3.1 There is an optimal solution y to LPd sm f{D, f,w) with the set C = {S C 
V(D) : y(S) > 0} being a cross- free family of V(D). 

Claim 7.3.1 implies the following restricted linear program, denoted by LP rc i sm f(D, f, w), 
has the same objective value as LPd sm f(D, f,w). 
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minimize 



sec 



subject to 



E 



E vis) 



> 



SeC:a&8 in (S) 



S&C:a&S out (S) 



y(S) 



> 



o 



VSeC 



As C is a cross-free family, one can show that the constrained matrix of LP r d sm f(D, f, w) 
is a network matrix; see Section 8.4.1 in Chapter 8. Since this is true for any set of costs 
w, by the result on LP's with network matrix constraints (Theorem 8.1.1), it follows that 
LP r dsmf(D, f,w) is integral, and hence the following result. 

Theorem 7.3.2 The linear program LPd sm f(D,f,w) is integral for any vector w £ Z m . 



In this section, we show some applications of Theorem 7.1.1, including the Lucchesi- 
Younger's theorem on directed cut cover, the polymatroid intersection theorem, and Nash- 
Williams' theorem on graph orientations. We remark that the main result in Section 6.2 
about minimum cost rooted fc-connected subgraphs was first proved by using Theorem 7.1.1. 



Let D = (V, A) be a directed graph. A subset C of A is called a directed cut if there exists 
a nonempty proper subset U of V with 5 m (U) = C and 5 out (U) = 0. Note that if a graph 
is strongly connected, then there is no directed cut in it. A directed cut cover (also known 
as a dijoin in the literature) is a set of arcs intersecting each directed cut. 

Lucchesi and Younger proved the following min-max theorem for the minimum size 
of a directed cut cover, which was conjectured by N. Robertson and by Younger. 

Theorem 7.4.1 (Lucchesi- Younger Theorem). Suppose D = (V, A) is not a strongly 
connected graph. Then the minimum size of a directed cut cover is equal to the maximum 
number of arc-disjoint directed cuts. 

We will derive the Lucchesi- Younger theorem from Theorem 7.1.1. The starting 
point is this simple observation. 

Proposition 7.4.2 Let C consists of all sets U such that the collection of arcs entering 
U forms a directed cut, i.e. C := {U C V \ / U / V and 5 out {U) = 0}. Then C is a 
crossing family of subsets. 



7.4 Applications of Submodular Flows 



7.4-1 Directed Cut Cover and Feedback Arc Set 
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To model the minimum directed cut cover problem as a submodular flow problem, 
we set f(U) = —1 for each U G C and d(a) = — oo, c(a) = 0, w(a) = 1 for each arc. Note 
that since S out (U) = for each U 6 C, the linear program LP sm f(D,f,w) simplifies to 
the following (after replacing the negation of the original flow variables by x below, hence 
transforming the maximization to a minimization problem): 

minimize ^ 

aeA 

subject to x(5 in {U)) > 1 VU CC 

x a > Va G A 

In this linear program, an optimal integer solution corresponds to a minimum directed cut 
cover, as the constraints ensure that every directed cut has an arc selected. On the other 
hand, the dual is 

maximize y(U) 

subject to ^2 V( u ) < 1 VaCi 

UeC:ae6 in (U) 

y{U) > VUeC 

An optimal integer solution to the dual linear program corresponds to a maximum collec- 
tion of disjoint directed cuts, as the constraints force the directed cuts to be arc-disjoint. 
By Theorem 7.1.1, both the primal and the dual programs have integral optimal solutions, 
and thus the Lucchesi- Younger theorem follows by the strong duality theorem of linear pro- 
gramming (Theorem 2.1.9). Furthermore, an efficient separation oracle for the minimum 
directed cut cover problem can be constructed via standard flow techniques, and thus the 
problem can be solved in polynomial time. 

7. 4-1-1 Feedback Arc Set in Directed Planar Graphs 

An interesting corollary of the Lucchesi- Younger theorem is a min-max relation for the 
minimum size of a feedback arc set in a directed planar graph. A feedback arc set in a 
directed graph D = (V, A) is a set of arcs intersecting every directed cycle. The problem 
of finding a minimum feedback arc set is NP-complete in general digraphs. For planar 
digraphs, however, the problem of computing a minimum feedback arc set can be reduced 
to the problem of computing a minimum directed cut cover. 

Given a directed planar graph D, its (directed) planar dual D* is constructed as 
follows. Let Go be the underlying undirected graph of D, and let G* D be its planar dual 
(a vertex in G* D is a face of Gd, and two vertices in G* D are adjacent if and only if the 
corresponding faces in Gd are adjacent). For each arc wx of D, let yz be the corresponding 
dual arc in G* D . The direction of yz is chosen so that it crosses the arc wx from left to right. 
Intuitively, the direction of yz is obtained by rotating the arc wx clockwise. The resulting 
directed planar graph is the planar dual D*. It can be easily seen that the directed cycles 
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Fig. 7.2. In this example, the directed planar graph D is shown in solid edges and its dual D' 
is dotted edges. The arcs (A, B) and (C, B) form a feedback arc set and the corresponding dual 
edges (a, b) and (6, c) form a directed cut cover in the dual graph. 



of D correspond to the directed cuts of D* . See Figure 7.2. Recall that a feedback arc set 
in D intersects every directed cycle in D. This corresponds to a set of arcs in the planar 
dual D* that intersects every directed cut in D* , which is by definition a directed cut cover 
in D* . Therefore, by computing a minimum directed cut cover in D*, a minimum feedback 
arc set can be constructed in D. Hence the minimum feedback arc set problem can be 
solved in polynomial time in directed graphs. Moreover, the Lucchesi- Younger theorem 
translates to the following beautiful min-max theorem. 



Theorem 7.4.3 Let D = (V, A) be a planar directed graph. Then the minimum size of a 
feedback arc set is equal to the maximum number of arc-disjoint directed cycles. 
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7. 4-2 Polymatroid Intersection 

Recall that a matroid is characterized by its rank function: In fact, it can be shown that r 
is the rank function of a matroid if and only if r is integer valued non-negative submodular 
function and r(T) < r(U) < \U\ if T C U. Given a matroid M, we have already seen (in 
Chapter 5) Edmonds' characterization of the independent set polytope of M, as well as 
that of the common independent sets in the intersections of two different matroids. 

The above results can be generalized to arbitrary submodular function starting from 
the observation that the rank function of a matroid is a specific submodular function. 
Given a submodular set function / on a ground set V, the polytope defined below is called 
the polymatroid associated with /, which is an integral polytope if / is an integer-valued 
function. 

x s >0 for s £ S, 
x(U) < f(U) forUCS 

Similarly, for two submodular set functions fa, fa on S, the following system charac- 
terizes the polymatroid intersection polytope LP pm i: 

x s > for s £ S, 
x{U) < fa{U) iovUCS 
x{U) < fa{U) iovUCS 

Notice that matroid intersection is a special case since the rank function of a matroid is 
submodular. The theorem below is due to Edmonds. 

Theorem 7.4.4 If fa and fa are submodular, then the polymatroid intersection polytope 
and its dual are integral. 

This result can be derived from Theorem 7.1.1. First a directed graph D = (V, A) 
is constructed as follows. Let S' and S" be two disjoint copies of S, let V = S' U S", and 
A = {s"s' | for s € S}. Note that D = (V, A) is a directed matching, where each arc in A 
corresponds to an element in the ground set S. Define C = {U' \ U C S}U{S'UU" | U C S} 
where U' and U" denote the sets of copies of elements of U in S' and S" , and define 
/ : C -)• M+ by 

f(U'):=h(U) toiUCS, 
f(V\U"):=fa(U) iovUCS, 
f(S') :=mm{f 1 (S),fa(S)}. 

Then C and / satisfy the crossing submodular condition. That is, C is a crossing 
family and / is a crossing submodular function since fa and fa are submodular functions. 
See Figure 7.3. If we take d = and c = oo, the submodular flow problem becomes the 
polymatroid intersection problem (for each U, the constraint for U' ensures that x(U) < 
fa(U) and the constraint for V \ U" ensures that x(U) < fa{U)). So Theorem 7.4.4 
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Fig. 7.3. In this example, the graph D is shown along with two typical sets U\ and U 2 in C. 

follows from Theorem 7.1.1. As in the case of matroid intersection, a min-max theorem 
on polymatroid intersection follows from Theorem 7.4.4. The proof is the same as in the 
proof of Theorem 5.4.5. 



Theorem 7.4.5 Let f\ and ji be submodular set functions on S with /i(0) = /2(0) = 0. 
Then 

max x(U) = min(/i(T) + f 2 {U \ T)). 



xeLP, 



TCU 



Furthermore, the following discrete separation theorem can be proved, using a similar 
construction as in the proof of Theorem 7.4.4. We defer the details to the exercises. 



Theorem 7.4.6 Let S be a ground set, p:2 s -^ZU {— oo} a supermodular function and 
b : 2 s — > Z U {oo} a submodular function for which p(0) = 6(0) = and p < b. Then there 
exists an integer-valued modular function m for which p < m < b. 



7.4-3 Graph Orientation 

Recall that a directed graph is strongly fe-arc-connected if there are k arc-disjoint paths 
between every ordered pair of vertices. Given an undirected graph, when does it have a 
strongly /c-arc-connected orientation? Nash- Williams proved the following necessary and 
sufficient condition. 
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Theorem 7.4.7 An undirected graph G has a k- strongly- connected orientation if and only 
if G is 2k- edge- connected. 



We derive Nash- Williams theorem using Theorem 7.1.1. The submodular flow prob- 
lem is defined on directed graphs; to reduce graph orientation to submodular flow, we first 
choose an arbitrary orientation D of G. If D is strongly fc-arc-connected, then there is 
nothing to prove. Otherwise, the task is to reverse the orientations of some arcs to obtain 
a strongly /c-arc-connected orientation. For each arc a there is a variable x a , where x a = 1 
means that the direction of a will be reversed, while x a = means that the direction of a 
will be retained from the arbitrary initial orientation. 

Let ^ U C V be an arbitrary subset. After reversing the directions of some arcs, 
the target is to have k arcs entering each set U (and thus by Menger's theorem, the resulting 
directed graph is strongly /c-arc-connected). Before reversing, there are d l £(U) arcs entering 
U. The number of new arcs entering U after reversing is x(5 out (U)) —x(S m (U)). Therefore, 
to have k arcs entering U after reversing, the constraint is d l £(U)— x{5' bn {U))-\-x{5 out {U)) > 
k, and thus the goal is to find an integral vector x satisying: 

x{5 in {U))-x{5 out {U)) < dfi(U)-k, 
< x a < 1. 



Note that the right hand side d l ^j(U) — k is a crossing submodular function, and thus 
this "re-orientation" problem is a submodular flow problem. Observe that when G is 2k- 
edge-connected, feasible fractional solution to this submodular flow problem, 

since 



d^{U) + x{5T{U))-x(^S(U)) 



\d in {U) + \d out {U) 



> \(2k 
= k. 



d°D{U)) + ^d out (U) 



By Theorem 7.1.1, the linear program for the submodular flow problem is integral. Since 
there is a feasible solution when G is 2fc-edge-connected, there is also an integral solution 
when G is 2/c-edge-connected, proving Nash- Williams' Theorem 7.4.7. 

Note that by formulating the orientation problem as a submodular flow problem, 
many generalizations can be solved consequently. One example is the weighted version 
where the two possible orientations of an arc may have different costs and the goal is to 
find the minimum cost strongly /c-arc-connected orientation. This also includes a special 
case of orientating mixed graphs, where the directions of some edges are fixed. 
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Another example is the degree-constrained version where the indegree (and thus the 
outdegree) of each vertex is fixed. To see this, consider the following extra constraints: 

x{5 in {v))-x{5 ou \v)) < d{v) 
x{5 in {V -v))-x{5 out {V -v)) < d(v) 

These force the difference between d in (v) and cP n *(t>) to be fixed, and so the resulting 
orientation must have d m (v) = d l £(v) — d(v), where d^(v) is the indegree in the initial 
orientation. These extra constraints are only defined on singletons and complement of 
singletons, and one can verify that the resulting function is still crossing submodular no 
matter what d(v) is. We note that the degree-constrained orientation problem (even with- 
out connectivity requirement) already captures many interesting combinatorial problems 
such as bipartite matching, score sequences of tournament and basketball league winner 
problems. 



7.5 Minimum Bounded Degree Submodular Flows 

In this section we consider the minimum bounded degree submodular flow problem, which 
is a generalization of the submodular flow problem we just addressed earlier. In this 
problem we are given a directed graph D = (V,A), a crossing submodular set function 
/ : 2 V -»• ZU {+00}, a node set W C V, and a function g : W — > Z + . A degree- constrained 
0-1 submodular flow is a vector x 6 {0, with the following properties: 

x(5 in (S)) - x(5 out (S)) < f(S) for every S^V, 

x(5(v)) < g(v) for every v £ W, 

where S(v) = 5 m (v) U 5 out (v). If W = 0, then this is the well-studied submodular flow 
problem we encountered earlier (Even though we have a minimization problem rather than 
the maximization version we introduced earlier, the transformation between the two is 
direct by negating the weights w) . However, the addition of the degree constraints makes 
the feasibility problem NP-complete. We show the following by adapting the iterative 
proof of Theorem 7.1.1 in Section 7.2. 

Theorem 7.5.1 There exists a polynomial time algorithm for the minimum bounded degree 
submodular flow problem which returns a 0-1 submodular flow of cost at most OPT that 
violates each degree constraint by at most one, where OPT is the cost of an optimal solution 
which satisfies all the degree constraints. 



7.5.1 Linear Programming Relaxation 

The proof of Theorem 7.5.1 is based on the natural linear programming relaxation for the 
problem which we denote LPf )( i srn f(D,f,w,g). 
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minimize w a x a 

subject to x(5 in (S)) -x(5 out (S)) < f(S) VSC]/ 

x(5(v)) < g{v) \/v G W 

< x a < 1 VaGi 

As before, this linear program can be solved by using an algorithm for minimizing 
submodular functions as a separation oracle. 



7.5.2 Characterization of Extreme Point Solutions 

The following characterization of the extreme point solutions follows from Lemma 7.2.3 
and the Rank Lemma. 

Lemma 7.5.2 Let x be any extreme point solution to LPbd S mf(D, f,w,g) with < x a < 1 
for each arc a G A(D). Then there exist a cross-free family C and a set T C W such that 

(i) x{5 in {S)) - x(5 out {S)) = f(S) for each S G C and x(S(v)) = g(v) for each veT. 

(ii) The vectors in {x{S) '■ S G C} U {x(3( v )) '■ v £ T} are linearly independent. 
(hi) |C| + |T| = |A(£>)|. 



7.5.3 Iterative Algorithm 



Before we go into the iterative algorithm for the minimum bounded degree submodular 
flow problem, let us first note that separate indegree and outdegree constraints can be 
incorporated into the submodular flow problem. This is achieved by a "node-splitting" 
technique, which will be useful in the iterative algorithm. Suppose we are given an indegree 
bound Bl n and an outdegree bound B° ut on a vertex v. To incorporate these constraints, 
we split the vertex v into two new vertices v m and v out , such that an arc uv is replaced by 



uv 



and an arc vu is replaced by v out u. See Figure 7.4 for an illustration. 
The set function / is modified as follows: 

B in 



f{S) 



f(S) 

f(s- 

oo 



{v 



m ,v out } + v) 



if S = {V in }, 

if S = V - {v out } 

if sn{v in ,v out } : 

if {v 

otherwise 



m X Mi } c s, 



It can be verified that /' is a crossing submodular function if / is a crossing submodular 
function, and thus the new problem is also a submodular flow problem. By construction, 
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Fig. 7.4. The node-splitting operation. 



v m has no outgoing arcs, and so the submodular flow constraint x(S m (v m )) — x(5 out (v m )) < 
f'(v m ) reduces to the indegree constraint x(S m (v m )) < B™. Similarly, v out has no incom- 
ing arcs, and so the submodular flow constraint x(5 m (V — v out )) — x(5 out (V — v out )) < 
f'iV — v out ) reduces to the outdegree constraint x(5 out (v out )) < B° ut . Therefore, a feasible 
submodular flow in this new problem is a feasible submodular flow in the original problem 
satisfying the indegree and outdegree constraints on v. 

With this node-splitting technique, we are ready to describe the iterative algorithm 
for the minimum bounded degree submodular flow problem. 



Iterative Minimum Bounded Degree Submodular Flow Algorithm 

(i) Initialize F = and /' <- f. 

(ii) While A / do 

(a) Compute an optimal extreme point solution x to LPbd S mf(D,f',w,g). 
Delete all arcs a with x a = 0. 

(b) For each arc a = uv with x a = 1, delete a from D, add a to F, decrease 
g{u) and g(v) by 1, and update f'(S) <- f(S) - df{S) + d%?(S) for each 
SCV. 

(c) (Relaxation 1) For each vertex v 6 W with d{v) < g(v) + 1, remove v 
from W . 

(d) (Relaxation 2) For each vertex v G W with g{y) = 1, remove v from 
W, add an indegree constraint x{5 in {v)) < 1 and an outdegree constraint 
x(5 out (v)) < 1. 

(iii) Return the solution x. 



Fig. 7.5. Minimum Bounded Degree Submodular Flow Algorithm 
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7.5.4 Correctness and Performance Guarantee 

The iterative algorithm is similar to that for the submodular flow problem, but with two 
relaxation steps. The relaxation step in (ii) (c) is the usual step of removing a degree 
constraint on v when d(v) < g{v) + 1, and so the degree violation on v is at most one. 
The relaxation step in (ii) (d) is a new step, where a degree constraint on v is replaced by 
an indegree constraint and an outdegree constraint on v, which can be incorporated into 
the submodular flow problem via the node-splitting technique. Observe that the current 
solution corresponds to a feasible solution of this relaxation. Since x(5 in (v)) < 1 and 
x(S out (v)) < 1, this means that the degree bound on v is violated by at most 1. Note that 
using the node-splitting technique the size of the graph is increased by 1 . However, the 
number of arcs is never increased and \W\ decreases, so the number of steps is polynomial. 
Hence Theorem 7.2.4 would follow if the algorithm always terminates successfully. 

A counting argument is used to prove that the algorithm always terminates. Each 
arc is assigned two tokens, for a total of 2\A\ tokens. For each arc a, one token is assigned 
to each endpoint. We shall show that if none of the steps in the algorithm can be applied, 
then each set in C and each degree constraint in T can collect two tokens, and there are 
some tokens left. This would imply \A\ > \C\ + |T|, which contradicts that x is an extreme 
point solution. 

By Step (ii) (d) and Step (ii) (c) of the algorithm, we may assume that g{v) > 2 and 
d(v) > g(v) + 2 for every v £ W. Hence d(v ) > 4 for v £ W, and thus each degree 
constraint has two extra tokens. The remaining counting argument is very similar to that 
of Lemma 7.2.6, and so only the difference is highlighted here. The proof is by induction 
on the height of the subtree defined by the laminar family (constructed from the cross-free 
family as in Section 7.2.4). If a set S is the smallest set containing a vertex v with degree 
constraint, then S can collect two tokens from the extra tokens on v. Otherwise, the same 
argument as in Lemma 7.2.6 will give two tokens to S. This shows that each set in C and 
each degree constraint in T can collect two tokens. Finally, the same argument as in the 
last paragraph of Lemma 7.2.5 will prove that there is some unused token, a contradiction. 
Therefore the algorithm always terminates, and this completes the proof of Theorem 7.5.1. 

7.5.5 Applications 

We close with some applications of the minimum bounded degree submodular flow problem. 

Minimum Bounded Degree Directed Cut Cover 

Let D = (V, A) be a directed graph. Recall that a set of vertices X is called a directed cut 
if 5 out {X) = 0, and a subset of arcs F is called a directed cut cover if \F fl 5(X)\ / for 
every directed cut X (recall that S(X) = 5 in (X) U 5 out {X) = 5 in (X) for such sets X). In 
the minimum bounded degree directed cut cover problem, we are given a directed graph 
D = (V,A), a cost function c : A — > Z, and a degree bound g(v) for each v £ V. The 
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task is to find a directed cut cover FCiof minimum cost such that \FP\ S(v)\ < g{v) for 
every v € V. As seen in Section 7.4, the directed cut cover problem can be reduced to the 
submodular flow problem. Thus Theorem 7.5.1 implies the following result. 

Corollary 7.5.3 There exists a polynomial time algorithm for the minimum bounded degree 
directed cut cover problem which returns a directed cut cover F of cost at most OPT and 
\F Pi 5(v)\ < g(v) + 1 for each vertex v €zV, where OPT is the cost of an optimal solution 
which satisfies all the degree constraints. 

Minimum Bounded Degree Graph Orientation 

In the minimum bounded degree graph orientation problem, we are given a directed graph 
D = (V,A), a cost function c : A — > Z, and a degree bound g(v) for every v G V. The 
task is to find an arc set of minimum cost whose reversal makes the digraph strongly 
/c-arc-connected, with the additional constraint that the number of arcs reversed at each 
node v is at most g(v). As seen in Section 7.4, graph orientation problems (with cross- 
ing supermodular requirements) can be reduced to the submodular flow problem. Thus 
Theorem 7.5.1 implies the following result. 

Corollary 7.5.4 There exists a polynomial time algorithm for the minimum bounded degree 
graph orientation problem which finds an arc set of cost at most OPT whose reversal makes 
the digraph strongly k- arc- connected and such that the number of arcs reversed at each node 
v is at most g(v) + 1, where OPT is the cost of an optimal solution which satisfies all the 
degree constraints. 



Notes 

The polyhedral result on submodular flows is due to Edmonds and Giles [49], generaliz- 
ing the result by Lucchesi and Younger [117] on directed cut cover and the polymatroid 
intersection theorem by Edmonds [47]. This paper by Edmonds and Giles also introduced 
the notion of totally dual integrality (TDI) and demonstrated the power of the uncrossing 
technique, which was developed earlier by Lucchesi and Younger [117] and Lovasz [115]. 

The starting point of graph orientation problems is the result by Robbins [143] 
showing that a graph has a strongly connected orientation if and only if it is 2-edge- 
connected. This result is extended by Nash- Williams [128] to obtain Theorem 7.4.7 and 
a further generalization on local edge-connectivities. The reduction of graph orientation 
problems to the submodular flows problem is due to Frank (see [56] for a survey). The 
result on degree constrained submodular flow problem is from Kiraly et al [97]. Gabow [68] 
also solves a degree constrained orientation problem using iterative rounding. 
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Exercises 



7.1 Fill in the details of the proof of Lemma 7.2.2. 

7.2 Prove Claim 7.3.1. 

7.3 (a) Generalize the proof of the integrality of the LP formulation for the submodular 

flow problem when box constraints are present, 
(b) Generalize the proof of the integrality of the dual of submodular flow when 
the primal has box constraints. (Hint: Observe that upper and lower bound 
introduce extra variables on arcs. Argue separately with tight arc constraints for 
which these variables are zero and non-zero.) 

7.4 Prove Theorem 7.4.5. 

7.5 Prove the discrete separation theorem (Theorem 7.4.6) using a construction similar 
to that for the polymatroid intersection theorem 7.4.5. 

7.6 Verify that the degree-constrained graph-orientation problem introduced in Sec- 
tion 7.4.3 by verifying that the extra degree constraints introduced there still result 
in a submodular function in the right-hand side of the constraints. 

7.7 Dilworth's Theorem states that in any partially ordered set, the size of the max- 
imum anti-chain is equal to the minimum number of chains needed to cover the 
whole set. Show that Dilworth's theorem is a special case of the submodular flow 
problem. 

7.8 (Frank and Tardos [58]) We outline the reduction of the minimum cost rooted k- 
connected subgraph problem (in Chapter 6) to the submodular flow problem. The 
proof is divided into two main steps. 

(a) Let G = (A, B; E) be a simple bipartite graph. Let p be an intersecting su- 
permodular function, and c a non-negative cost function on the edges. Call a 
subset ROE supporting if Tr(X) > p(X) for every X C A, where Tr(X) : = 
{y G B | yx G R for some x G A}. Prove that the minimum-cost supporting 
set of a bipartite graph can be found in polynomial time via a reduction to the 
submodular flow problem. 

(b) Let D = (V, E) be a directed graph. For sets X C V and FOE denote 
Of{X) := {u £ V — X : there is a uv G F with v G X}. Let p be an intersecting 
supermodular function, and c a non-negative cost function on the edges. We call 
a subset F C E of edges out- supporting if |(9f(X)| > p{X) for every X C V. 
Use (a) to prove that the minimum-cost out-supporting set of a digraph can be 
found in polynomial time. 

(c) Conclude from (b) that the minimum cost rooted fc-connected subgraph problem 
can be solved optimally in polynomial time. 

7.9 Show that the minimum bounded degree submodular flow problem can be solved 
in polynomial time if there are only degree constraints on the in-degree and the 
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out-degree of nodes. In particular, show that the method outlined in Section 7.5.3 
works by showing that the modified function /' is indeed submodular. 

7.10 What do the results in this chapter (mainly Theorem 7.5.1) imply for the feedback 
arc set problem in directed planar graphs? 
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8 

Network Matrices 



t 

In this chapter, we consider a simple model, based on a directed tree representation 
of the variables and constraints, called network matrices. We show how this model as well 
as its dual have integral optima when used as constraint matrices with integral right-hand 
sides. Finally we show the applications of these models, especially in proving the integrality 
of the dual of the matroid intersection problem in Chapter 5, as well as the dual of the 
submodular flow problem in Chapter 7. 

While our treatment of network matrices is based on its relations to uncrossed struc- 
tures and their representations, they play a crucial role in the characterization of totally 
unimodular matrices which are all constraint matrices which yield integral polytopes when 
used as constraint matrices with integral right hand sides [150]. Note that total unimodu- 
larity of network matrices automatically implies integrality of the dual program when the 
right hand sides of the dual are integral. 

The integrality of the dual of the matroid intersection and submodular flow polyhedra 
can be alternately derived by showing the Total Dual Integrality of these systems [150]. 
While our proof of these facts use iterative rounding directly on the dual, there is a close 
connection between these two lines of proof since both use the underlying structure on 
span of the constraints defining the extreme points of the corresponding linear program. 



8.1 The Model and Main Results 

Let T = (V, E(T)) be a directed tree (where all the arcs do not necessarily point towards a 
root - hence it is a weakly directed spanning tree, or a spanning tree on V in the undirected 
sense) and let D = (V, E(D)) be a directed graph on the same vertex set V. 

Define TV, an |-E(T)| x \E(D)\ matrix in the following way for arcs e G E(T) and 

t © Copyright 2011 by Lap Chi Lau, R. Ravi and Mohit Singh. Published 2011 by Cambridge 
University Press. 
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e3 








-1 


-1 


-1 


e 4 








-1 









(a) (b) 



Fig. 8.1. In Figure (a), tree T is shown in bold edges and graph D in dotted edges. In Figure (b), 
the corresponding network matrix is shown. 



/ € E(D). Let P be the path from tail of / to the head of / in T. See Figure 8.1 for an 
example. 



/ (0 if e does not occur on P \ 

N e f = < +1 if e occurs in the forward direction on P 
\ — 1 if e occurs in the reverse direction on P j 

Such {0, +1, —1} matrices arising from a given directed tree T and a given directed 
graph D on the same set of vertices are called network matrices. 

A well-known result in combinatorial optimization is the integrality of linear pro- 
grams with network constraint matrices. 

Theorem 8.1.1 Let N be a network matrix and b be an integral vector. Then the extreme 
point solutions of the linear program {maxc x : Nx < b} are integral. 

This follows from the result that network matrices are actually a subclass of a broader class 
of constraint matrices that have integral extreme point solutions, called totally unimodular 
matrices. The latter are matrices that have all square sub-determinants equal to 0, +1 or 
— 1, and permit several equivalent rich characterizations - for more on them, see the books 
by Schrijver [149, 150]. 

In this chapter, we use an iterative method to prove both the above theorem as well 
as its dual version below, and discuss some applications of these theorems. 
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Theorem 8.1.2 Let N be a network matrix and c be an integral vector. Then the extreme 
point solutions of the linear program {mmb T y : y T N > c T } are integral. 

8.2 Primal Integrality 

Following our framework we start with the natural linear programming relaxation for the 
problem. 

8.2.1 Linear Programming Relaxation 

For a tree arc e £ E(T), denote 1(e) as the set of arcs in E(D) whose tail-head paths in 
T contain e and are oriented in the same direction as e, and O(e) as the set of arcs whose 
tail-head paths in T also contain e but are oriented in the opposite direction as e. We 
denote the following linear program by LP net (T, D). 

< b e Ve££(T) 

> o yfeE(D) 

8.2.2 Characterization of Extreme Point Solutions 

For an arc e 6 E(T), the corresponding inequality ^/e/(e) x f ~ E/eO(e) x f — be defines a 
characteristic vector x(e) £ M} E ^ D ^: the vector has an 1 corresponding to each arc / 6 /(e), 
a —1 corresponding to each arc / 6 0(e), and otherwise. The following lemma follows 
directly from the Rank Lemma. 

Lemma 8.2.1 Let x be any extreme point solution to LP ne t(T, D) with non-zero Xf for 
each arc f £ E(D). Then there exists a subset R C E(T) such that 

(i) £/6/(e) X f ~ E/eO(e) X f = b e f 0r each e G K 
(ii) The vectors in {x( e ) '■ e £ -R} are linearly independent, 
(m) \R\ = \E(D)\. 

8.2.3 Iterative Algorithm 

To devise an iterative algorithm, we look for a variable set to an integral value, and then 
modify the network representation appropriately by deleting this element and updating 
the constraint right hand sides appropriately. The algorithm is shown in Figure 8.2. 



maximize Cf-Xf 
feE(D) 

subject to x f~ x f 

fel(e) feO(e) 

x f 
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Iterative Algorithm for Network Constrained Matrix Linear Programs 

(i) While E(D) / do 

(a) Find an optimal extreme point solution x to LP net (T, D). 

(b) Fix an arc / = uv G E(D) with Xf = a/ for an integer a/. For every 
constraint e G E(T), update b e <— b e + sign(f,e) • aj where sign(f,e) is 
— 1 if / € /(e), +1 if / G 0(e) and otherwise. Delete the arc / from D, 
and its corresponding column from the constraint matrix. 

(ii) Return the integral solution x. 



Fig. 8.2. Algorithm for Network Constrained Matrix Problems 

8.2.4 Correctness and Optimality 

A simple verification shows that while fixing the variables in the iterative algorithm, the 
new constraint matrix is a network matrix as well. Indeed, for an arc / set to an integer 
value, along with the appropriate changes in the right hand side to keep the residual 
program feasible, the network representation of the remaining variables is induced by the 
original one with the arc / removed from the directed graph D, while the tree T remains 
unchanged. Thus, assuming the algorithm terminates successfully, a standard inductive 
argument shows that the returned solution has optimal (maximum) cost. 

Theorem 8.2.2 The iterative algorithm for Network Constrained Matrix Linear Programs 
in Figure 8.2 returns an optimal integral solution in polynomial time. 

The following lemma shows that the algorithm will terminate and completes the 
proof of Theorem 8.2.2. 

Lemma 8.2.3 For any extreme point solution x to LP ne t(T, D), there is an arc f G E{D) 
with integer value, i.e. Xf = aj for an integer aj. 

The proof follows closely the proof of the similar lemma (Lemma 7.2.5) for the 
submodular flow problem. Suppose for contradiction that for each arc / G D, we have 
that Xf is not integral. From Lemma 8.2.1, we start with a maximal set of tight constraints 
that are linearly independent at the extreme point solution x, and denote the corresponding 
subset of arcs by R. Note that every arc in D has Xf > since there are no integer valued 
variables. We will show a contradiction to \R\ = \E(D)\ as stated in Lemma 8.2.1. 

First, for ease of description, consider contracting the arcs in E(T) \ R to get the 
reduced tree T' . Let V' denote the vertices in the contracted graph. We update the arcs 
in E(D) by contracting the vertices accordingly, and call the resulting directed graph D' . 
Observe that this contraction does not affect any constraint for an arc in R = E(T'). We 
again use a counting argument to derive a contradiction: For each arc in E(D') correspond- 
ing to a nonzero variable in x, one token is assigned to its head and one token is assigned 
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to its tail. So the total number of tokens assigned is exactly 2\E(D')\. These tokens will 
be redistributed such that each arc e G E(T') is assigned two tokens, and there are still 
some excess tokens left. We use a simple token assignment scheme: every arc in E(T') gets 
the tokens collected by its lower endpoint in the rooted tree. We then show that the root 
of the tree also has some tokens. Thus we have \E(D)\ > \E(D')\ > \E(T')\ = \R\, giving 
the desired contradiction. The following lemma shows that the redistribution works. 

Lemma 8.2.4 For the tree T' with root r, the tokens assigned to vertices in the tree can 
be distributed such that every arc in the subtree gets at least two tokens and the root gets 
at least two tokens. 

Proof First consider a leaf arc in the tree corresponding to a tight constraint. The nonzero 
variables contributing to this arc must have an endpoint at the lower end of this leaf arc. 
Since the right hand side of this arc's constraint is integral and no nonzero variable is 
integral, there must be at least two nonzero variables corresponding to arcs participating 
in this constraint, and hence at least two arcs from D' incident on the lower endpoint of 
this leaf arc as required. 

Next, consider an internal arc e = (u,v) £ T' oriented towards the root with lower 
endpoint u (if it is oriented the other way, the argument is symmetric by switching the 
signs appropriately). Let the arcs below e incident on u in T' be e\, e<i . . . e&. Consider the 
tight constraints corresponding to e and subtract from it, those corresponding to e\, . . . , e^: 

k k 

(2 x f~ Yl x f )-^2dir(ei,e)- ( ^ x f - ^ x f ) = b e - ^ dir(e h e) • b ei , 
fel(e) feO(e) i=l fel{ei) feO( ei ) i=l 

where we define dir(ei,e) to be +1 if both are oriented in the same direction (along the 
path to the root) and —1 otherwise. Note that in the above sum we are only including 
the nonzero a £ D' . If any arc a originates in a vertex below u in one of the subtrees and 
ends in another vertex below u in a different subtree, its two contributions in the left hand 
side get canceled out. Also, if an arc a 6 D' has one endpoint in a vertex below u and the 
other above it, then again, its two terms get canceled out in the left hand side. Thus the 
only arcs whose contributions survive in the left hand side in the above equation are those 
that have one endpoint in u, which are precisely those that give their tokens to u. 

Now the left hand side of the above equation cannot be zero since in that case, we 
have linear dependence between the tight constraints for arcs e, ei, . . . , e^, contradicting 
Lemma 8.2.1. Thus the left hand side has a nonzero variable. But the right hand side is 
an integer, and all the variables x a in the left hand side are non-integral. Therefore, there 
must be at least two variables in the left hand side, thus giving two tokens to the endpoint 
u as required. 

To get extra tokens at the root of T', note that we can always root the tree T' at a 
leaf, and the argument we provided above for a leaf applies to the root as well, giving it 
at least two extra tokens, and hence the desired contradiction. □ 
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This completes the proof of Lemma 8.2.3. 



8.3 Dual Integrality 

We now look at linear programs which are dual to network constrained problems, and argue 
their integrality as well. Note that the constraint matrices of these dual linear programs 
are transposes of network matrices. 



8.3.1 Linear Programming Relaxation 

We start with the given linear programming relaxation for the problem. As before, for a 
tree arc e G E(T), denote /(e) as the set of arcs in D whose tail-head paths in T contain 
e and are oriented in the same direction as e, and O(e) as the set of arcs whose tail-head 
paths in T also contain e but are oriented in the opposite direction as e. Here Cf is integral 
for each / G E(D). 

minimize b e • y e 

edE{T) 

subject to ^2 y e ~ ^2 Ve > c f V / G E(D) 

e:/6/(e) e:feO(e) 

Ve > VeG£(T) 

This is the dual of LP ne t(T, D) in the previous section, and we denote it by LPd ne t(T, D). 
We proceed as in the previous section to prove the integrality of LPd ne t(T, D), and hence 
Theorem 8.1.2. 



8.3.2 Characterization of Extreme Point Solutions 

For an arc / G E(D), the corresponding inequality S e :/e/(e) U e ~ Se:/eO(e) Ve > c f defines 
a characteristic vector x(f) £ R^"^: the vector has an 1 corresponding to each arc e with 
/ G /(e), a —1 corresponding to each arc e with / G 0(e), and otherwise. The lemma 
below follows directly from the Rank Lemma. 

Lemma 8.3.1 Let y be any extreme point solution to LPd ne t(T, D) with non-integral y e 
for each arc e G E(T). Then there exists a subset D' C E{D) such that 

(i) Ee:/e/(e) Ve ~ Ee:/gO( e ) Ve = c f f° r each f ^ D ' '■ 

(ii) The vectors in {x(f) '■ f 6 D'} are linearly independent. 
(hi) \D'\ = \E(T)\. 
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8.3.3 Iterative Algorithm 



To devise an iterative algorithm, as before, we look for a variable set to an integral value, 
and then modify the network representation appropriately by deleting this element and 
updating the constraint right hand sides appropriately. 

Iterative Algorithm for Duals of Network Constrained Matrix LP 

(i) While E(T) / do 

(a) Find an optimal extreme point solution y to LP^ net (T, D). 

(b) Fix an arc e = uv G E(T) with y e = a e for an integer a e . For every 
constraint / G E(D), update c/ <— c/ + sign(e, f) • a/ where sign(e, /) is 
— 1 if / G /(e), +1 if / G 0(e) and otherwise. Contract the arc e from 
T, and delete the corresponding column from the constraint matrix. 

(ii) Return the integral solution y. 



Fig. 8.3. Algorithm for Duals of Network Constrained Matrix Problems 



8.3.4 Correctness and Optimality 

A simple verification shows that the update steps in the above algorithm lead to a constraint 
matrix which is again a transpose of a network matrix, while the directed tree T and the 
directed graph D are modified by contracting the vertices {u, v} corresponding to the arc 
e = {uv } fixed. Thus assuming the algorithm terminates successfully, it is straightforward 
to verify that the returned solution has optimal (minimum) cost. 

Theorem 8.3.2 The Iterative Algorithm for Duals of Network Constrained Matrix linear 
Programs in Figure 8.3 returns an optimal integral solution in polynomial time. 

The following lemma shows that the algorithm will terminate. 

Lemma 8.3.3 For any extreme point solution y to LP<i net (T ', D) , there is an arc e G E(T) 
with integer value, i.e. y e = a e for an integer a e . 

Suppose for contradiction, y e is not an integer for any arc e G E(T). From Lemma 8.3.1, 
we start with a maximal set of tight constraints that are linearly independent at the ex- 
treme point solution y, and denote the corresponding subset of arcs by D' . Observe that 
y e > for each e G T since no variable is integral. We will show a contradiction to 
\D'\ = \E(T)\ as stated in Lemma 8.3.1. 

We use a counting argument to derive a contradiction: The arcs in D' corresponding 
to tight constraints in y also form a graph over the vertices in V(T). First we argue that 
the graph formed by D' is acyclic. Since \E(T)\ = \D'\ and T is a tree, this implies that D' 
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also forms a tree. Then we use the path in D' between the endpoints of an arc e G E(T) 
to show that the variable y e must be set to an integer value, giving us the lemma. 

In detail, first suppose for a contradiction that there is a cycle C (in the undirected 
sense) among the arcs in D' . Fix an orientation of the cycle and let the arcs encountered 
in this order be fi, /2, . . . , fk (not necessarily all oriented in the same direction as the cycle 
C). Consider the weighted sum of constraints corresponding to fi, - ■ ■ ,fk- 

k 

i=l e:/i6/(e) e:/;eO(e) 

where sign(fi) is +1 if it is oriented in the same direction as C and —1 otherwise. Switching 
the order of summation this is the same as 

Y Ve '( Si 9 n (fi)~ Y Si 9 n (fi))- 

eeE(T) fiel(e) /ieO(e) 

It is now easy to verify that since the arcs fi form a cycle, the terms for every tree arc e 
cancel out in the above summation to reduce it to zero, thus showing that a cycle of tight 
constraints in D' are linearly dependent, contradicting Lemma 8.3.1. Thus, there are no 
cycles in D', and hence D' is also a tree. 

Finally, consider an arc e 6 E(T) going from a leaf u G T to its parent v (the case 
when it is pointed to the leaf can be argued symmetrically). Since D' is a tree, there is an 
(undirected) path from u to v in D\ say, fi, f'2, ■ ■ ■ , f P - Again, consider the sum of tight 
inequalities 

p ^ p 

^2sign(fi)( Ve~ Yl V^) = ^2 c fv 

i=l e:fi£l(e) e:fi€0(e) i=l 

where sign(fi) is +1 if it is oriented in the same direction as the path from u to v and —1 
otherwise. Switching the summation to be over the arcs e G E(T), we can now see that 
this sum simplifies to either plus or minus the single term y e . On the other side, the right 
hand sides of the constraints corresponding to the arcs fi, ■ ■ ■ , f p are all integers, and so 
we finally get that y e equals either plus or minus this sum which is an integer, as claimed. 

This completes the proof of Lemma 8.2.3. 

8.4 Applications 

Traditional applications of the integrality of network matrices range from the integrality 
of vertex cover and maximum matching problems in bipartite graphs to its generalization 
to the max-flow min-cut theorem in digraphs. In this section, we use the framework of 
network matrices to show integrality of the dual LP's for various problems we encountered 
earlier, such as the maximum matroid basis problem and the maximum matroid intersection 
problem in Chapter 5, as well as the submodular flow problem in Chapter 7. 
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8.4-1 Matroid 



In this section, we show the integrality of the dual linear program for the maximum weight 
basis problem as discussed in Section 5.4.1 and prove Theorem 5.4.2 which we state again. 

Theorem 8.4.1 The linear programming formulation LPd ma t(M) is integral. 

Proof Let y be an optimal solution to LPd mat (M). Claim 5.4.1 implies that we can assume 
that the support of y is a chain C. Moreover, the following restricted linear program, 
denoted by LP r d ma t(M), has the same objective value as LPd ma t(M). 

minimize r(T)y(T) 
Tec 

subject to y{T) > w e Ve £ S 

TeCieGT 

y(T) > VTGC 

In the following lemma, we show the constraint matrix of LPd ma t(M) is the transpose 
of a network matrix when C is a chain. 

Lemma 8.4.2 Given a set S, let C be a chain over the subsets of S. Then the \S\ x \C\ 
matrix N, with the entry (e, R) for e £ S and R 6 C is 1 if and only if e 6 R, is the 
transpose of a network matrix. 

Proof Construct a directed tree T with a root vertex r and one vertex for each set in C. 
There is an arc from R 6 C to R' G C if and only if R' is the smallest set containing R. 
Moreover there is an arc from the set R to root r where R is the largest set in C. Thus T 
is a directed path towards r. 

The graph D is defined over the same vertex set. For each element e in S which is contained 
in some set of C, there is an arc from the vertex corresponding to the smallest set containing 
e to the root r. See Figure 8.4. It is easy to check that the network matrix corresponding 
to (T, D) is exactly the transpose of the matrix N. □ 

Theorem 5.4.2 now follows from Theorem 8.1.2. □ 



8.4-2 Matroid Intersection 

In this section, we show the integrality of the dual linear program for the maximum weight 
common independent set problem as discussed in Section 5.4.2 and prove Theorem 5.4.4 
which we state again. 

Theorem 8.4.3 The linear programming formulation LPdi n t(Mi, M?) is integral. 
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(a) 



Fig. 8.4. In Figure (a), the chain C = {Si,S 2 , S 3 , £4} is shown where u e S 4 and v e S 2 - In Figure 
(b), the corresponding tree has a vertex r and one vertex for every set Si. Each element in S 
corresponds to a non-tree arc from the smallest set containing it to r. Hence /„ is from S4 to r 
and /„ is from S 2 to r. 

Proof Let y be an optimal solution to LPdi nt (Mi, M 2 ). Claim 5.4.1 implies that we can 
assume that the set d = {T C S : yi(T) > 0} and the set C 2 = {T C S : y 2 {T) > 0} are 
chains over the ground set S. Moreover, the following restricted linear program, denoted 
by LP r di n t(M\, M2), has the same objective value as LPd int (Mi, M2). 

minimize n(T) yi (T) + r 2 (T)y 2 (r) 

TeCi Tec 2 

subject to + ^2 U2{T) > w e Ve G S 1 

TeCi:eST TeC 2 :eST 

yi(T) > VT G Ci, 1 < i < 2 

In the following lemma, we show the constraint matrix of LP r( n nt (M) is the transpose 
of a network matrix when C\ and C 2 are chains. 

Lemma 8.4.4 Given a set S, let C\ and C 2 be chains over the subsets of S. Consider the 
\S\ x (|C|i + IC2I) matrix N, where the entry (e, R) for e 6 S and R £ C\U C 2 is 1 if and 
only if e 6 R. Then N is the transpose of a network matrix. 

Proof Construct a directed tree T with a vertex r along with one vertex for each set in C\ 
or C 2 . For each set R in C\, there is an arc with its tail in R and its head in the smallest 
set containing R in C\. If R is the largest set in C\ then the head of the arc is the root 
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r. For each set R in C2, there is an arc with its tail in R and its head in the largest set 
contained in R. For the largest set in C2 the tail of the arc is the root r. Thus the tree T 
is a directed path towards the smallest set in Ci- 

We now define the graph D with the same vertex set as T. For each element of S which 
is contained in some set of C\ or C2, there is an arc from the smallest set in C\ containing 
e to the smallest set in C2 containing e. If no set in C\ or C2 contains e, then the head or 
the tail is changed to r accordingly. See Figure 8.5. 



Fig. 8.5. In Figure (a), the chains C\ — {Si, S2, S3, S4} and C2 = {Ri, R2, R3} are shown where 
u e S4 n i?2 and v £ S2 n Ri. In Figure (b), the corresponding tree has a vertex r and one vertex 
for every set in Ci U C2 ■ Each element in S corresponds to a non-tree arc from the smallest set 
containing it in Ci to the smallest containing it in C2. Hence /„ is from S4 to R2 and f v is from 
^2 to Ri. 

Again it is straightforward to check that the network matrix corresponding to (T, D) is 
the transpose of the matrix N. □ 

Theorem 5.4.4 now follows from Theorem 8.1.2. □ 



In this section, we show that the dual linear program of the submodular flow problem is 
integral by reducing the constraint matrix to a network matrix as discussed in Section 7.3 
and prove Theorem 7.3.2 which we state again. 





(b) 



8.4-3 Submodular Flows 



Theorem 8.4.5 The linear program LPd sm f{D, f,w) is integral for any vector w G Z 
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Proof Let y be an optimal solution to LPd S mf{D, f, w). Claim 7.3.1 implies that we can as- 
sume that the support of y is a cross-free family C. Moreover, the following restricted linear 
program, denoted by LP r d sm f(D,f,w), has the same objective value as LPd S mf(D, j \w). 

maximize f(S) y{S) 

sec 

subject to ^2 v( s ) ~ ^2 y( s ) < w a Va € A 

SeC:aeS in (S) SeC:ae<S°«*(S) 

y{S) > \/SeC 

We leave it as an exercise to verify that the the constraint matrix of LP r d S mf(D, f, w) 
is the transpose of a network matrix when C is a cross- free family. (This also follows directly 
from Corollary 13.21a of Schrijver [150].) □ 



Notes 

A matrix A is called totally unimodular if every square submatrix has determinant in 
{0, +1, —1}. It is easy to see that if the constraint matrix of a linear program is totally 
unimodular and the right hand side of the constraints is integral then the linear program is 
integral. Total unimodularity is a very strong tool for proving integrality of linear programs 
for many optimization problems. The fact that network matrices are totally unimodular 
was proved by Tutte [158]. Total dual integrality (TDI), introduced by Edmonds and 
Giles [49], is a weaker condition than total unimodularity but is also very powerful tool in 
establishing integrality of linear programs for optimization problems. Indeed, almost all 
integrality results discussed in this book have been proven to be integral showing that the 
given linear programming description is also totally dual integral (see also [150]). 



Exercises 

8.1 Perhaps the most famous application of the fact that network matrix constraints 
result in integer solutions for LPs is to show the max-flow min-cut theorem: Given 
a directed graph with nonnegative capacities on the arcs, and a source node s and 
a sink node t, the maximum flow problem asks for the maximum set of disjoint 
paths from s to t obeying the capacities on the edges (i.e. the maximum number 
of paths using an arc is at most its capacity) . 

Formulate a LP relaxation for the maximum flow problem with the property 
that its constraint matrix is a network matrix. (Hint: Write one constraint per 
node except for t denoting flow conservation at all nodes except s, and observe 
that the constraint matrix is the node-arc incidence matrix of the digraph minus 
one row; show that such matrices have a network representation). 
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8.2 One can formulate a linear program for the max-flow problem by first converting 
it into a circulation problem: namely, adding an extra arc from t to s and now 
insisting that the flow is conserved at all nodes (in-flow equals out-flow everywhere) . 
To enforce a certain flow, we can add lower and upper bound constraints on the 
flow through all the arcs in the digraph. Formalize this equivalence and show that 
this extended formulation also continues to have integral extreme points by casting 
it as a network matrix constrained problem. 

8.3 Show using the integrality of the duals of network constrained matrices and strong 
LP duality, that the maximum s — t flow in a digraph is the capacity of a minumum 
s — t cut. 

8.4 Prove Theorem 8.4.5. (Hint: represent the cross-free family as a tree as in the proof 
of Theorem 7.2.4 in Section 7.2.4.) 
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9 

Matchings 



t 

Given an weighted undirected graph, the maximum matching problem is to find a 
matching with maximum total weight. In his seminal paper, Edmonds [45] described an 
integral polytope for the matching problem, and the famous Blossom Algorithm for solving 
the problem in polynomial time. 

In this chapter, we will show the integrality of the formulation given by Edmonds [45] 
using the iterative method. The argument will involve applying uncrossing in an involved 
manner and hence we provide a detailed proof. Then, using the local ratio method, we 
will show how to extend the iterative method to obtain approximation algorithms for the 
hypergraph matching problem, a generalization of the matching problem to hypergraphs. 



9.1 Graph Matching 

Matching in bipartite graph are considerably simpler than matchings in general graphs; 
indeed, the linear programming relaxation considered in Chapter 3 for the bipartite match- 
ing problem is not integral when applied to general graphs. See Figure 9.1 for a simple 
example. 



9.1.1 Linear Programming Relaxation 

Given an undirected graph G = (V, E) with a weight function w : E — > 1Z on the edges, 
the linear programming relaxation for the maximum matching problem due to Edmonds 
is given by the following LPm(G). Recall that E(S) denotes the set of edges with both 
endpoints in S C V and x(F) is a shorthand for ^eeF Xe ^ or ^ C. E. 

t © Copyright 2011 by Lap Chi Lau, R. Ravi and Mohit Singh. Published 2011 by Cambridge 
University Press. 
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Fig. 9.1. In this example, if the weight of the cut edge is small, then the above fractional solution 
is an optimal solution to the linear program with only degree constraints. 



maximize 



subject to 



w P x P 



x{5{v)) < 1 



x(E(S)) < 



\s\-i 



> 



Vi> G V 

VScV, \S\ odd 
Ve G E 



Although there are exponentially many inequalities in LPm{G), there is an efficient 
separation oracle for this linear program, obtained by Padberg and Rao using Gomory-Hu 
trees. We refer the reader to the exercises for a description of the separation algorithm. In 
this section we prove the following main theorem. 



Theorem 9.1.1 Every extreme point solution to the linear program LPm(G) is integral. 



9.1.2 Characterization of Extreme Point Solutions 

We prove the following crucial lemma characterizing extreme point solutions to LPm(G). 
Again the uncrossing technique and the Rank Lemma form the basis of the argument, but 
there are also some parity arguments involved since the constraints are only defined on 
odd-sets. For a subset of arcs F C E, it defines a characteristic vector x(-F) G with 
an 1 corresponding to each edge in F and otherwise. 

Lemma 9.1.2 Let x be an extreme point solution to LPm{G) with < x e < 1 for each 
edge e G E{G). Then there exists a laminar family C of odd- sets and a set of vertices 
T C V such that 

(i) x(E(S)) = (\S\ - l)/2 for each S G C and x(6(v)) = 1 for each v G T. 
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(ii) The vectors in {x(E(S)) : S 6 £■} U {x(^( v )) '■ v £ T} are linearly independent. 

(iii) E(S) is connected for each set S € C. 

(iv) \E\ = \C\ + \T\. 

To prove Lemma 9.1.2, we first prove the following claim about tight odd-sets, which 
are the sets S with \S\ odd and x(E(S)) = (\S\ - l)/2. 

Lemma 9.1.3 If A,B are tight odd-sets and A D B ^ 0, then one of the following is true: 

(i) An B and A U B are tight odd-sets and x( E (A)) + x( E (B)) = x( E (A n B)) + 
X(E(ADB)). 

(ii) ^4 \ B and B\ A are tight odd-sets, the degree constraints in An B are tight, and 
X(E{A)) + X (E(B)) = X (E(A \ B)) + X (E(B \ A)) + Z veA nB *(<*(«))■ 

Proof First consider the case that |An5| is odd. Since \A\ + \B\ = \AnB\ + \Al)B\, this 
implies that \A U B| is also odd. Hence, we have 

= x(E(A)) + x(E(B)) 

< x(E{AnB))+x{E(AuB)) 

\AHB\-1 \AUB\-1 

~ 2 + 2 

\A\-l \B\-l 

2^2' 

where the first inequality follows from supermodularity of the function l-E^S 1 )! (Proposi- 
tion 2.3.6), and the second inequality follows from the constraints of LPm(G). Hence both 
inequalities are satisfied at equality. The second inequality thus implies that An B and 
AU B are tight odd-sets, while the first inequality implies that there are no edges between 
A\B and B\A and thus x(E(A)) + x(E(B)) = x{E(A n B)) + x(E(A U B)). 

Next we consider the case when \A Pi B\ is even, in which case \A \ B\ is odd. Since 
\A\ + \B\ = \A\B\ + \B\A\ +2\AnB\, it follows that \B\A\ is also odd, and hence: 

\ A ^1+\ B ^1 = X (E(A)) + x(E(B)) 

< x(E(A\B)) + x(E(B\A))+ x(5(v)) 

veAnB 

\A\B\-1 |B\A|-1 „, 

< 1 —^y — + — + \adb\ 

\B\-1 
2 2' 

where the first inequality can be verified by counting the contribution of each edge, and 
the second inequality follows from the odd-set constraints of A \ B and B \ A and the 
degree constraints in A n B. Therefore both inequalities must hold as equalities. The 
second inequality thus implies that A \ B and B \ A are both tight odd-sets, and also all 
the degree constraints in A D B are tight. The first inequality implies that there are no 
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edges with one endpoint in A n B and another endpoint in V — (A U B), and therefore 
X (E(A)) + X (E(B)) = X {E{A \ B)) + X (E(B \ A)) + Z ve AnB x(S(v)). □ 



The next claim shows that a tight odd-set that is not connected can be "replaced" 
by a connected tight odd-set and some tight degree constraints. 

Claim 9.1.4 If S is a tight odd-set for which E(S) is not connected, then there exists 
a tight odd-set R C S such that R is a connected component in S, and 2x{E(S)) = 
2 X (E(R)) + ZveS\RX(S(v)) and x(5(v)) = 1 for each v G S \ R. 

Proof Since \S\ is odd, there exists a connected component R of E(S) with \R\ odd, and 

\S\-1 = 2x(E(S)) 

= 2x(E{R)) + 2x(E(S - R)) 

= 2x(E(R))+ Yj x(5(v)) - x(5(S - R)) 
ves-R 

< \R\ - 1 + \S-R\ -x(8(S-R)) 
= \S\-l-x(5{S-R)), 

where the second equality follows because R is a connected component, the third equality 
follows from the fact that YlveS\R x ($( v )) = ^(x(E(S — R))) + x(S(S — R)), and the 
inequality follows because of the odd-set constraint of R and the degree constraints for 
vertices in S — R. Hence the inequality must hold as an equality, which implies that R 
is a tight odd-set that is connected, and x(S(v)) = 1 for each v £ S \ R. Furthermore, 
we must have x(S(S — R)) = 0, which implies that 5(S — R) = and thus 2x(E(S)) = 
2 X (E(R)) + Z v es\RX(Hv))- ' □ 

With the uncrossing operations developed in Lemma 9.1.3 and Claim 9.1.4, we are 
now ready to prove Lemma 9.1.2. 

Proof of 9.1.2: The proof structure is similar to that of Lemma 4.1.5, but the details 
are more subtle. Let T = {S \ x(E(S)) = (\S\ — l)/2} be the set of tight odd-sets and 
T = {{v} | x(S(v)) = 1} be the set of tight vertices for an extreme point solution x to 
LPm(G). Denote by span{FUT) the vector space generated by the characteristic vectors 
in {x(E(S)) | S G J 7 } U {x(d( v )) I v £ T}. Let £ be a maximal laminar family of T 
with the additional property that each set S £ C is connected. Using Lemma 9.1.3 and 
Claim 9.1.4, we will prove that span(C U T) = span(FU T), and then the lemma follows 
directly from the Rank Lemma. 

Suppose, by way of contradiction, that span(CL)T) C span(F\JT). Recall that 
two sets A and B intersect if A D B, A — B, B — A are all non-empty. As in Lemma 4.1.5, 
we define inter sect(A, £) = \{B G C \ A and B are intersecting}), i.e. the number of sets 
in C that intersect A. Since span(C U T) C span{F U T), there exists a set A G T with 
x(E(A)) $l span(CL) T). Choose such a set A with minimum intersect(A, C) and then 
minimum \A\. We will show that x(E(A)) G span(CL)T), leading to a contradiction. 
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First suppose that inter sect{A, C) = 0, then A must be disconnected; otherwise CUA 
is also a laminar family with each set in CU A connected, contradicting the maximality of 
C. By Claim 9.1.4, there is a tight odd-set B which is a connected component in A, and 

2 X {E{A)) = 2 X (E(B))+ Y, x(S(v)), (9.1) 

v£A\B 

where every vertex in A\B is in T. By Claim 9.1.5 we have inter sect(B, C) < inter sect(A, C) = 
0. Since \B\ < \A\, by the choice of A, x(E{B)) G span(CUT), and hence by (9.1) it 
follows that x(E(A)) G span(CUT), a contradiction. 

Therefore intersect(A, C) > 1, and let B G C be a set that intersects A. Since A,B 
are tight odd sets, by Lemma 9.1.3, either An B and AU B are both tight odd sets, or 
A \ B and B \ A are both tight odd sets. First consider the case when An B and A U B 
are tight odd sets. By Claim 9.1.6, both intersect{AC\ B,C) and intersect(AL) B,£) are 
smaller than inter sect(A, £). Since A and B are both connected, AUB is also connected, 
and thus x(E(A U B)) G span(C U T) by the choice of A. If An B is also connected, 
then we have x(E(A n B)) G span(C U T) by the choice of A. Otherwise, if A fl i? is not 
connected, then by Claim 9.1.4 there is a tight odd-set R which is a connected component 
in A fl B such that 

2x(E(AnB)) = 2x(E(R))+ £ X (S(v)), (9.2) 

De(Ans)\R 

where every vertex in (A H B) \ R is in T. By Claim 9.1.5 and Claim 9.1.6 we have 
inter sect(R, C) < inter sect(An B, C) < inter sect(A, £), and thus x(E(R)) £ span(CUT) 
by the choice of A, and hence x(E(AnB)) G span(CUT) by (9.2). Therefore, in either case, 
wehavex^^nB)) G span(£uT). Since x(-E(A))+x(.E(£)) = x{E(AnB))+x(E(AuB)) 
by Lemma 9.1.3 and x(E(B)), x(E(AnB)), x(E(AUB)) are all in span(CUT), this implies 
that x(.E(.A)) G span(CUT) as well, a contradiction. 

Finally we consider the case when A\B and i?\^4 are tight odd sets. By Lemma 9.1.3, 
we have 

x(E(A)) + x(E(B)) = x(E(A\B)) + x(E(B\A))+ £ x (5(t,)), (9.3) 

where each vertex in ^4 n B is in T. By Claim 9.1.6 both intersect(A \ B,C) and 
inter sect{B \ A,C) are smaller than inter sect(A, £). If A \ B is connected, then \ 
B)) G span(C U T) by the choice of A. Otherwise, if A \ B is not connected, we 
can use a similar argument (using Claim 9.1.4) as in the previous paragraph to show 
that x{E(A \ B)) G span(C U T). Similarly we have x( E (B \ A)) G span(C U T). 
Since x(E(B)), X (E(A \ B)), X (E(B \ A)) are all in span(C U T), by (9.3) we also have 
x(E(A)) G span(CUT), a contradiction. 

To finish the proof of Lemma 9.1.2 it remains to prove the following two claims, 
whose proofs are deferred to the exercises. 
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Claim 9.1.5 If A is not connected and R is a connected component in A, then we have 
inter sect(R, C) < inter sect(A, £). 

Claim 9.1.6 If A ^ C and B € C are intersecting, then all the four numbers intersect(An 
B, C), inter sect(AU B, C), inter sect{A\ B, C) and intersect(B\A,£) are strictly smaller 
than inter sect(A, C). 



9.1.3 Iterative Algorithm 

The following is a simple iterative procedure which returns a matching of optimal cost and 
also shows that LPm(G) is integral. 



Iterative Matching Algorithm 




(i) Initialization F «— 0. 




(ii) While V(G) / do 




(a) Find an optimal extreme point solution x 


to LPm(G) and remove every 


edge e with x e = from G. 




(b) If there is an edge e = {u, v} with x e = 


1 then update F F U {e}, 


G^G\{u,v}. 




(iii) Return F. 





Fig. 9.2. General Matching Algorithm 



9.1.4 Correctness and Optimality 

Assuming that we can always find an edge e with x e = 1 in Step (ii) (b) of the algorithm, 
the returned solution is a maximum matching by using the by-now standard inductive 
argument. It remains to show that the algorithm always finds an edge e with x e = 1 in 
Step (ii)(b). 

Lemma 9.1.7 Given any extreme point x to LPm(G) there must exist an edge e with 
x e = or x e = 1. 

Proof Suppose for contradiction that < x e < 1 for each edge e 6 E. Let C be a laminar 
family of tight odd-sets and T be a set of tight vertices satisfying the properties given by 
Lemma 9.1.2. Let C = CUT be the extended laminar family. We show a contradiction to 
the fact that \E\ = \C\ + \T\ = \C'\ by a token counting argument. Initially, each edge is 
given one token, for a total of \E\ tokens. Each edge will give its token to the smallest set 
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in £ that contains both of its endpoints. Then we will redistribute the tokens inductively 
so that each member in £ receives one token and there are some tokens left. This will 
imply that \E\ > \£\ giving us the desired contradiction. 

Recall that a laminar family naturally defines a forest L as follows: Each node of L 
corresponds to a set in £ , and there is an edge from set S to set R if S is the smallest 
set containing R, in which case we say S is the parent of R and R is a child of S. In this 
problem, the number of tokens a set S G £ received is equal to the number of edges induced 
in S but not induced in the children of S. Henceforth, when we consider S G £ , it is more 
convenient to consider the graph Gs formed by contracting the children of S in G[S] into 
singletons. See Figure 9.3. For a vertex v G Gs, we let R v G £ be the corresponding child 
of S in G[S]. The following claim shows that Gs and G[S] have similar properties. 

Claim 9.1.8 For S G £, \G S \ is odd and x(E(G s )) = ^= i . 

Proof Let the children of S be R\, R2, ■ ■ ■ , Rk- Since each Ri is of odd cardinality, con- 
tracting it into a singleton does not change the parity of the size, and hence \Gs\ is odd 
since \S\ is odd. Also, since each Ri is a tight odd-set, we have 

x(E(G s )) = x(E(S)) - £ x(E(R t )) = - £ = IGS ^ 1 - 



□ 



We say a set S G £ is degree-tight if x(S(S)) = 1 and the characteristic vector 
X(S(S)) can be written as a linear combination of the characteristic vectors in the subtree 
of C rooted at S. Note that the tight vertices in T are the singleton degree-tight sets. The 
following claim gives a partial characterization when a set S £ £ — T is degree-tight. 

Claim 9.1.9 A set S G £ — T is degree-tight if every vertex v in Gs is degree-tight. 

Proof Since S G £, by Claim 9.1.8, we have 2x(E(G s )) = \G S \ - 1. Therefore x(5(S)) = 
x(S(G s )) = Z v eGs " ME(G S )) = \G S \ - (\G S \ - 1) = 1. Note that 

X (S(S)) = x(S(v)) - 2 X (E(G S )). (9.4) 

As each v is degree-tight, each x($( v )) can be written as a linear combination of the 
characteristic vectors in the subtree of L rooted at R v . Note also that x(E{Gs)) = 
x(E(S)) — YlveGs X{E(Rv))- Therefore, by (9.4), x(S(S)) can be written as a linear com- 
bination of the characteristic vectors in the subtree of L rooted at S, and hence S is a 
degree-tight set. □ 

The token is redistributed inductively by the following lemma, in which degree-tight 
sets play a special role. 

Lemma 9.1.10 For any rooted subtree of L with root S, the tokens assigned to edges 
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induced in S can be distributed such that every node in the subtree gets at least one token. 
Furthermore the root S gets at least one token unless S is degree-tight. 



Proof We prove the lemma by induction. Let the degree-tight children of S be R\, R2, . . ■ , Rk 
Note that if a child R of S is not degree-tight, then R has already received one token from 
the edges induced in R by the induction hypothesis. Let m := \E(Gs)\ be the number 
of edges in Gs- Since each edge in Gs gives one token to S, hence S has m tokens. If 
m> k + 1, then the tokens received by S can be redistributed so that each i?j receives one 
token and S receives one token, as desired. 

Hence we assume that m < k. Since G[S] is connected by Lemma 9.1.2, Gs is also 
connected and thus m > \Gs\ — 1- Suppose m = \Gs\- Then k > m = \Gs\ and so every 
vertex in Gs is degree-tight, and hence 5 is a degree-tight set by Claim 9.1.9. Therefore, 
by redistributing the m = k tokens of S to give one token to each Ri, the lemma follows. 

Henceforth we assume that m = \Gs\ — 1- Since Gs is connected, it implies that Gs is a 
tree, and thus is a bipartite graph. Let G\ and G2 be the bipartition of Gs- See Figure 9.3. 
Since \Gs\ is odd, we can assume |Gi| < | C2 1 , which implies that |Gi| < ^ S J 1 . 



S 




Graph G[S] Graph G s 



Fig. 9.3. In this example, the nodes R 2 and R4 are in G\ and R\,R% and R 5 are in G 2 . 
Since Gs is bipartite, we have E(Gs) Q U^eGi $(v), and thus 

= x(E(G s )) < < IGil < ^4"^' (9 - 5) 

veG-i 

where the second inequality of (9.5) follows from the degree constraints of vertices in G±. 
Thus equalities hold as equalities. The second inequality of (9.5) implies that x(5(v)) = 1 
for each v € G\, and it follows that 

x(S(S)) = x ( 6 ( v )) ~ ME(G(S))) = \G S \ - (\G S - 1) = 1. (9.6) 

veG s 
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The first inequality of (9.5) implies that x(E(Gs)) = YlveGi x(^{ v ))i an d so 

X (E(S)) = x(E(G s )) + E X (E(Rv)) = E *(*(«)) + E X(^))- (9.7) 

t>eG s i>GGi d£G s 

Recall that we assume that k > m = \Gs\ — 1- Suppose k = \Gs\. Then each vertex v 
in G\ is degree-tight, and therefore each x(3( v )) can be written as a linear combination 
of the characteristic vectors in the subtree of L rooted at R v . It follows from (9.7) that 
x(E(S)) can be written as a linear combination of the characteristic vectors in the subtree 
of L rooted at S (but not including S itself), contradicting the linear independence of the 
characteristic vectors in £' as described in Lemma 9.1.2. 

Therefore, since we assume k > m, the only case left is k = \Gs\ — 1, in which case exactly 
one vertex w in S is not degree-tight. If it; G G2, then every vertex v in G\ is degree- 
tight, then we would have the same contradiction as in the previous paragraph. Therefore, 
we assume w G G\, and thus every vertex in G2 is degree-tight. Since x(E(Gs)) = 
YlveG-i x(,$( v )) (see the sentence preceding (9.7)), we have 

X(S(S)) = E x{S{v)) - 2x(E(G s )) = E x(S(v)) - x(E(G s )). 
f GG1UG2 veG 2 

As each vertex v in G2 is degree-tight, x(8(v)) can be written as a linear combination of 
the characteristic vectors in the subtree of L rooted in R v . Note also that x(E(Gs)) = 
x(E(S)) — J2 v eG s x(E(R v )), and thus x{$(S)) can be written as a linear combination of 
the characteristic vectors in the subtree of L rooted at S. This and (9.6) imply that S is 
a degree-tight set. Therefore, by redistributing the m = k = \Gs\ — 1 tokens of S to give 
one token to each Ri, the lemma is satisfied. □ 

To complete the proof of Lemma 9.1.7, we consider the graph Gy formed by contract- 
ing the roots of £ in G into singletons. Let the degree-tight vertices of Gy be v±, . . . ,vi, 
and let R Vi be the corresponding odd-set of Vi in G. By Lemma 9.1.10 v±, . . . , v\ are the 
only vertices in Gy requiring a token. Each edge in Gy has an unassigned token; we let 
m := \E(Gy)\. To prove Lemma 9.1.7, it suffices to show that m > I. Since each Vi is 
degree-tight and there is no edge e with < x e < 1, each Vi is of degree at least two in 
Gy. Hence 

l 

2m = E >X>MI>2Z. 

v&Gv »=1 

Suppose for contradiction that m = I. Then the inequalities must hold as equalities. The 
second inequality implies that Gy is a disjoint union of cycles, while the first inequality 
implies that every vertex in the cycles is a degree-tight vertex. Consider a cycle C in 
Gy. Since every vertex in C is degree-tight, we have 2x(E(C)) = X^eG x ($( v )) = | C | - 
Therefore, C cannot be an odd cycle; otherwise IJ^eG is an odd set violating the cor- 
responding odd set constraint in LPm(G). So C is an even cycle, and let C\,Ci be the 
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bipartition of C. Then 

E *(*(«)) = E ( g - 8 ) 

As each vertex v G C is degree-tight, can be written as a linear combination of 

the characteristic vectors in the subtree of L rooted in R v . Hence (9.8) implies a linear 
dependence among the constraints in C, contradicting Lemma 9.1.2. Therefore m > I, 
and this implies that \E\ > \C'\, completing the proof. □ 



9.2 Hypergraph Matching 

A hypergraph H = (V, E) consists of a set V of vertices and a set E of hyperedges, where 
each hyperedge e G E is a subset of vertices. A subset M C E{H) of hyperedges is a 
matching if every pair of hyperedges in M has an empty intersection. Given a hypergraph, 
a weight w e on each hyperedge e, the hypergraph matching problem is to find a matching 
with maximum total weight. The graph matching problem is the special case when every 
hyperedge has exactly two vertices. A hypergraph is called k-uniform if every hyperedge 
has exactly k vertices. A hypergraph H is called k-partite if H is fc-uniform and the set 
of vertices can be partitioned into k disjoint sets V±, V2, ■ ■ ■ , Vk so that each hyperedge 
intersects every set of the partition in exactly one vertex. Note that a bipartite graph is a 
2-partite hypergraph. The main result of this section is the following theorem. 

Theorem 9.2.1 For the hypergraph matching problem, there is a polynomial time (k — 
1 + ^-approximation algorithm for k-uniform hypergraphs, and a (k — 1)- approximation 
algorithm for k-partite hypergraphs. 

We prove Theorem 9.2.1 for 3-partite hypergraphs. This is also known as the 3- 
dimensional matching problem, one of the classical NP-complete problem. The generaliza- 
tions to /c-uniform and fc-partite hypergraphs are deferred to the exercises. 



9.2.1 Linear Programming Relaxation 

We use the standard linear programming relaxation for the hypergraph matching problem. 
In the following 5(v) denotes the set of hyperedges that contains v. 



maximize w e x e 

subject to x e < 1 Vt> G V 

eeS(v) 

x e > V e G E 



150 



For the analysis of the iterative algorithm, we consider a slightly more general linear 
program, denoted by LPm(H,B), where B denotes the vector of all degree bounds < 
B v < 1 for each vertex v £ V. Initially B v = 1 for each v G V . 



B v \/ v G V 

Ve G E 



9.2.2 Characterization of Extreme Point Solutions 

The lemma below follows by a direct application of the Rank Lemma 1.2.3. 

Lemma 9.2.2 Given any extreme point solution x to LPm(H, B) with x e > for each 
e G E, there exists W C V smc/i i/iai 

(i) x(<S(v)) = B v > /or eac/i v G W. 

(ii) T/ie vectors in {x(3( v )) '■ v 6 ^} are linearly independent. 

(iii) |W| = |^|. 

9.2.3 Iterative Algorithm and Local Ratio Method 

The algorithm consists of two phases. In the first phase we use an iterative algorithm to 
provide a "good" ordering of the hyperedges. In the second phase we apply the local ratio 
method to this good ordering to obtain a matching with cost at most twice the optimum. 
The algorithm is presented in Figure 9.4. In the following let N[e] be the set of hyperedges 
that intersect the hyperedge e. Note that e G N[e\. 

We remark that if the problem is unweighted, then there is a direct iterative rounding 
algorithm; see exercises. Also, after we have the ordering, there is an alternative (but 
inefficient) rounding method based on a greedy coloring procedure. See the exercises at 
the end of this chapter for more details about these two remarks. 

The Local- Ratio routine described below provides an efficient procedure to obtain 
a 2-approximate solution for the 3-dimensional matching problem. 

To prove the correctness of the algorithm, we show that the iterative algorithm 
always succeed in finding an ordering with a "good" property. Then, using the property of 
the ordering, we prove that the local ratio method will return a matching with cost at least 
half the optimum. The good property of the ordering is defined in the following theorem. 

Theorem 9.2.3 After Step (ii) of the iterative algorithm in Figure 9.4, there is an ordering 



maximize w e x e 

subject to x e < 

ee6(v) 

X e > 
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Iterative 3-Dimensional Matching Algorithm 

(i) Find an optimal extreme point solution x to LPm(H,B) with B v = 1 for all v. 
Initialize F «— 0. 

(ii) For i from 1 to \E(H)\ do 

(a) Find a hyperedge e with x(N[e\) < 2. 

(b) Set fi <- e and F <- F U {/<}. 

(c) Remove e from i7. 

(d) Decrease B v by x e for all v £ e. 

(iii) M •(— Local- Ratio(F, w), where w is the weight vector of the hyperedges. 

(iv) Return M. 

Fig. 9.4. 3-Dimensional Matching Algorithm 



Local-Ratio(F, w) 

(i) Remove from F all hyperedges with non-positive weights. 

(ii) If F = 0, then return 0. 

(iii) Choose from F the hyperedge e with the smallest index. Decompose the weight 
vector w = w\ + W2 where 

Mel) _ f „(«) it^ivH, 

I otherwise. 

(iv) M' <- Local-Ratio (F,w 2 ). 

(v) If M' U {e} is a matching, return M' U {e}; else return M' . 

Fig. 9.5. The Local Ratio Routine 

of the hyperedges with x(N[ei] D {e^, ej+i, . . . , e m }) < 2 /or all 1 < i < m, where m is the 
number of hyperedges in x with positive fractional value. 

The proof of Theorem 9.2.3 consists of two steps. First, in Lemma 9.2.4, we prove 
that there is a hyperedge e with x(N[e\) < 2 in an extreme point solution to LPm(H, B). 
Since the initial solution x is an extreme point solution, this implies that the first itera- 
tion of Step (ii) of the iterative algorithm will succeed. Then we prove in Lemma 9.2.5 
that the remaining solution (after removing e and updating B v ) is still an extreme point 
solution to LPm(H, B). Therefore, by applying Lemma 9.2.4 inductively, the iterative 
algorithm will succeed in finding an ordering of hyperedges {ei, . . . ,e m } with x(N[ei\ D 
{ej, ej+i, . . . , e m }) < 2 for all 1 < i < m. Now we prove Lemma 9.2.4. 

Lemma 9.2.4 Suppose x is an extreme point solution to LPm(H,B). If x e > for all 
e £ E, then there is a hyperedge e with x(N[e\) < 2. 
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Proof Let W be the set of tight vertices as described in Lemma 9.2.2. To show that there 
is a hyperedge with the required property, we first prove that in any extreme point solution 
to LPm(H, B) there is a vertex in W of degree at most two. Suppose for contradiction 
that every vertex in W is of degree at least three. This implies that 

\ W \ = \ E \ = ^vjM > E v ew\^)\ > m 
3 3 

where the first equality follows from Lemma 9.2.2, the second equality follows because every 
hyperedge contains exactly three vertices. Hence the inequalities must hold as equalities. 
In particular the first inequality implies that every hyperedge is contained in W. Let 
Vi, V2, V3 be the tri-partition of V, and W{ = W H Vi for 1 < i < 3. Since each hyperedge 
intersects Wi exactly once, we have 

vew-i vew 2 

This implies that the characteristic vectors in W are not linearly independent, contradicting 
Lemma 9.2.2. Therefore there is a vertex u G W of degree at most two. Let e = {u, v, w} 
be the hyperedge in S(u) with larger weight. Since u is of degree at most two, this implies 
that 2x e > x(5(u)). Therefore, 

x(N[e]) < x(5(u)) + x(5(v)) + x(S(w)) - 2x e < x(5(v)) + x(S(w)) < B v + B w < 2. 

□ 

The following lemma allows Lemma 9.2.4 to be applied inductively to complete the 
proof of Theorem 9.2.3. 

Lemma 9.2.5 In any iteration of Step (ii) of the algorithm in Figure 9.4, the restriction 
of the fractional solution is an extreme point solution to LPm{H,B). 

Proof Suppose the graph in the current iteration is H = (V, E). Let xe be the restriction 
of the initial extreme point solution x to E. We prove by induction on the number of 
iterations that xe is an extreme point solution to LPm{H,B). This holds in the first 
iteration by Step (i) of the algorithm. Let e = {^1,^2,^3} be the hyperedge found in 
Step (ii)(a) of the algorithm. Let E' = E - e and H' = {V,E'). Let B' be the updated 
degree bound vector. We prove that xe 1 is an extreme point solution to LPm{H' ,B'). 
Since the degree bounds of ^1,^2,^3 are decreased by exactly x e , it follows that xe> is 
still a feasible solution. Suppose to the contrary that Xe> is not an extreme point solution 
to LPm(H' ,B'). This means that Xe' can be written as a convex combination of two 
different feasible solutions y\ and j/2 to LPm(H' , £>'). Extending y\ and 2/2 by setting the 
fractional value on e to be x e , this implies that xe can be written as a convex combination 
of two different feasible solutions to LPm(H,B), contradicting that xe is an extreme point 
solution. Hence xe> is an extreme point solution to LPm(H' , B'). □ 

Now we use the local ratio method to obtain an efficient approximation algorithm 
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for the 3-dimensional matching problem. We need the following basic result of the local 
ratio method. 

Theorem 9.2.6 (Local Ratio Theorem [11]) Let C be a set of vectors in W 1 . Let 
w, W\,W2 £ M. n be such that w = w± + W2- Suppose x £ C is r- approximate with respect to 
w\ and r- approximate with respect to W2- Then x is r-approximate with respect to w. 

Using the ordering in Theorem 9.2.3, we prove the performance guarantee of the 
approximation algorithm. Note that by construction the local ratio routine returns a 
matching. It remains to prove that the cost of the returned matching is at least half the 
optimum. 

Theorem 9.2.7 Let x be an optimal extreme point solution to LPm(H,B). The matching 
M returned by the algorithm in Figure 9.5 satisfies w(M) > \ ■ w ■ x. 

Proof The proof is by induction on the number of hyperedges having positive weights. 
The theorem holds in the base case when there are no hyperedges with positive weights. 
Let e be the hyperedge e chosen in Step (iii) of the algorithm. Since e has the smallest 
index in the ordering, by Theorem 9.2.3, we have x(N[e\) < 2. Let w,w±,W2 be the 
weight vectors computed in Step (iii) of the algorithm. Let y' and y be the characteristic 
vectors for M' and M obtained in Step (iv) and Step (v) respectively. Since w(e) > 
and W2(e) = 0, W2 has fewer hyperedges with positive weights than w. By the induction 
hypothesis, W2 • y' > \ ■ W2 ■ x. Since W2(e) = 0, this implies that u>2 • y > \ ■ w>2 ■ x. 
By Step (v) of the algorithm, at least one hyperedge in N[e] is in M. Since x(N[e\) < 2 
and wi(e') = w(e) for all e' G N[e], it follows that w± ■ y > \ ■ w\ ■ x. Therefore, by 
Theorem 9.2.6, we have w ■ y > \ ■ w ■ x. This shows that M is a 2-approximate solution 
to the 3-dimensional matching problem. □ 



9.2.4 Partial Latin Square 

As an application, we show that the partial Latin square problem can be directly reduced 
to the 3-dimensional matching problem. A partial Latin square of order n is an n-hy-n 
array whose cells are empty or contain one color from {1, . . . , n}, with the restriction that 
each color appears at most once in each row and at most once in each column. For a partial 
Latin square L, the number of non-empty cells in L is denoted by \L\, and the color of a 
non-empty cell in row i and column j is denoted by L(i,j). For two partial Latin squares 
L and L' ', we say L ^ L' if L(i,j) = L'(i,j) for each non-empty cell in L. Given a partial 
Latin square L, the partial Latin square extension problem is to find a partial Latin square 
L' with L < L' and \L'\ — \L\ maximized. 

We show an approximation ratio preserving reduction from the partial Latin square 
extension problem to the 3-dimensional matching problem. Given a partial Latin square 
L of order n, we construct a tri-partite hypergraph H = (V, E) with 3n 2 vertices. Let 
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Vxy,Vxz,Vyz be the tri-partition of V. In V xy there is one vertex XiUj corresponding to 
each pair of row i and column j for all 1 < i, j < n. Similarly, in V xz there is one vertex 
XiZk corresponding to each pair of row i and color k for all 1 < i, k < n, and in V yz there 
is one vertex UjZk corresponding to each pair of column j and color k for all 1 < j, k < n. 
There is a hyperedge e = {xiUj,XiZk,yjZk} in H if the cell in row i and column j is 
empty and the color k does not appear in row i or column j. There is an one-to-one 
correspondence between matchings of size m in H and valid extensions of m cells for L. 
Indeed, if e = {xi]jj,XiZk, UjZk} is in the matching, then we fill the cell in row i and column 
j by color k, and vice versa. 



Notes 

The work of Edmonds [45] first showed the polyhedral characterization for the general 
matching problem. It forms the first class of polytopes whose characterization does not 
simply follow just from total unimodularity, and its description was a "break-through 
in polyhedral combinatorics" [150]. Cunningham and Marsh [41] showed that Edmonds' 
description is actually totally dual integral. In the same paper [45] Edmonds also presented 
the famous Blossom algorithm for solving the maximum matching problem in polynomial 
time. 

Most existing algorithms for the 3-dimensional matching problem are based on local 
search [88, 6, 31, 16]. Hurkens and Schrijver [88] gave a (| + e)-approximation algorithm 
for the unweighted problem, and Berman [16] gave a + e)-approximation algorithm for 
the weighted problem. The result presented in this chapter is by Chan and Lau [30], giving 
an algorithmic proof for the results by Fiiredi [63] and Fiiredi, Kahn and Seymour [64]. 
There is a local search 1.5-approximation algorithm for the partial Latin square extension 
problem [83]; the reduction in Section 9.2.4 shows that the result in [83] can be derived 
from the result in [88]. 



Exercises 

9.1 Verify Claim 9.1.5 and Claim 9.1.6. 

9.2 Adapt the arguments in this chapter to show an integral formulation for the maxi- 
mum weight fe-matching problem of choosing a set of k independent edges of max- 
imum weight in a given undirected graph. In particular, argue that it suffices 
to add one equality constraint insisting that the sum of chosen edges is k to our 
formulation for maximum matchings to achieve this. 

9.3 In an instance of the T-join problem, we are given a graph G = (V, E) with weight 
function w : E — >■ 1Z + and a set T of even cardinality, the task is to find a subgraph 
H such that the degree of v in H, dn{v) is even if v £ T and odd if v G T. 
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(a) Show that if T = V then the optimal T-join is the minimum weight perfect 
matching. 

(b) Generalize the linear programming formulation for the perfect matching 
problem to obtain a formulation for the T-join. 

(c) Generalize the argument for the perfect matching problem to obtain inte- 
grality of the linear programming formulation for the T-join problem. 

9.4 (Padberg and Rao [133]) A Gomory-Hu (cut-equivalent) tree for graph G = (V,E) 
with weight function w on edges is a tree T with vertex set V and a new weight 
function w' on edges of T. The edges of T can be distinct from the edges of G (i.e., 
T need not be a subgraph of G). The tree satisfies the following properties. 

(i) For every pair of vertices u, v, the weight of minimum (u, v)-cut in G (w.r.t 
w) is equal to the weight of the minimum (u,v)-cut in T (w.r.t w'). 

(ii) For each edge e G E(T), w'(e) is the weight of the cut Sg(S) where the 
removal of e from T results in components S and V \ S. 

Given a weighted undirected graph, a Gomory-Hu tree for it can be constructed in 
polynomial time (see e.g. [37] for details). Use this fact to prove the following. 

(a) Show that the minimum weight odd cut (which has odd number of vertices 
on either side) can be obtained using the Gomory-Hu tree. 

(b) Obtain an efficient separation oracle for the perfect matching problem in 
general graphs. 

(c) Can you use the ideas above to also supply an efficient separation oracle for 
LPm(G) for the maximum matching problem in general graphs? 

9.5 Let T = {S C V : \S\ is odd and x*(6(S)) = 1} where x* is an extreme point of 
LPm(G). Solving the linear program using the ellipsoid method enables us to get 
a set family JCf such that constraints for sets in T are independent and span 
all the tight constraints in T . But T need not be laminar. Give a polynomial time 
algorithm that, given T, returns a laminar family £Cf such that constraints for 
sets in £ are independent and span all the tight constraints in T ■ 

9.6 Modify Lemma 9.2.4 to prove that there is an edge e with x(iV[e]) < k — 1 + \ for k- 
uniform hypergraphs, and an edge e with x(N[e\) < k—1 for /c-partite hypergraphs. 
Use this to generalize the 3-dimensional matching result to prove Theorem 9.2.1. 

9.7 Show that the algorithm in Figure 9.4 can be modified to give a direct iterative 
rounding 2-approximation algorithm (without using the local-ratio routine) for the 
unweighted problem. 

9.8 This exercise is to develop an alternative proof that the integrality gap of the 3- 
dimensional matching problem is at most 2. Given a hypergraph H with fractional 
value x e on each edge, a fractional hyperedge coloring is a set of weighted matchings 
{Mi, . . . , Mi} of H so that Yli-eeM- w (^i) = x e- A fractional hyperedge coloring is 
called a fractional hyperedge fc-coloring if the total weight on the matchings is equal 
to k. Use a greedy coloring algorithm on the ordering in Theorem 9.2.3 to obtain 
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a fractional hyperedge 2-coloring of the hypergraph. Conclude that the integrality 
gap of the 3-dimensional matching problem is at most 2. 

9.9 Consider the following colorful graph matching problem. Given a undirected graph 
in which each edge e has weight w e and a color c(e), the colorful graph matching 
problem is to find a maximum weighted matching in which each color appears 
at most once. Extend the argument in this chapter to obtain a 2-approximation 
algorithm for this problem. 

9.10 A projective plane is a hypergraph which satisfies the following properties: 

(a) Given any two vertices, there is a unique hyperedge that contains both of 
them. 

(b) Given any two hyperedges, there is exactly one vertex contained in both of 
them. 

(c) There are four vertices such that no hyperedge contains more than two of 
them. 

It is known that a r-uniform projective plane exists if r — 1 is a prime power (see e.g. 
[119]). A truncated r-uniform projective plane is obtained by removing a vertex 
and all of its incident hyperedges from a r-uniform projective plane. Prove that a 
fc-uniform projective plane has integrality gap k—l+j-, and a truncated /c-uniform 
projective plane has integrality k — 1 for the linear programming relaxation for the 
hypergraph matching problem. 

9.11 Can you demonstrate any application of the weighted hypergraph matching problem 
to the partial Latin square extension problem? 
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10 

Network Design 



t 

In this chapter we study the survivable network design problem. Given an undirected 
graph G = (V, E) and a connectivity requirement r uv for each pair of vertices u, v, a Steiner 
network is a subgraph of G in which there are at least r uv edge-disjoint paths between u and 
v for every pair of vertices u, v. The survivable network design problem is to find a Steiner 
network with minimum total cost. In the first part of this chapter, we will present the 
2-approximation algorithm given by Jain [91] for this problem. We will present his original 
proof, which introduced the iterative rounding method to the design of approximation 
algorithms. 

Interestingly, we will see a close connection of the survivable network design problem 
to the traveling salesman problem (TSP) . Indeed the linear program, the characterization 
results, and presence of edges with large fractional value are identical for both problems. 
In the (symmetric) TSP we are given an undirected graph G = (V, E) and cost function 
c : E — ^ R + and the task is to find a minimum cost Hamiltonian cycle. In the second 
part of this chapter, we will present an alternate proof of Jain's result, which also proves 
a structural result about extreme point solutions to the traveling salesman problem. 

In the final part of this chapter, we consider the minimum bounded degree Steiner 
network problem, where we are also given a degree upper bound B v for each vertex v £ V, 
and the task is to find a minimum cost Steiner network satisfying all the degree bounds. 
Using the idea of iterative relaxation, we show how to extend Jain's technique to this more 
general problem. We will first present a constant factor bicriteria approximation algorithm, 
and then show how to improve it to obtain additive approximation guarantee on the degree 
violation. 



t © Copyright 2011 by Lap Chi Lau, R. Ravi and Mohit Singh. Published 2011 by Cambridge 
University Press. 
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10.1 Survivable Network Design Problem 

The survivable network design problem generalizes the minimum Steiner tree problem, the 
minimum Steiner forest problem, and the minimum fc-edge-connected subgraph problem. 
Hence the results in this chapter also applies to these problems. 

10.1.1 Linear Programming Relaxation 

To formulate the problem as a linear program, we represent the connectivity requirements 
by a skew supermodular function. As stated in Definition 2.3.10, a function / : 2 V — > Z is 
called skew supermodular if at least one of the following two conditions holds for any two 
subsets S, T C V. 

f(S) + f(T) < f(SUT) + f(SnT) 
f(S) + f(T) < f(S\T) + f(T\S) 

It can be verified that the function / defined by f(S) = rnax u€ s,v^s { r uv} for each subset 
S C V is a skew supermodular function; see Exercise 2.10. Thus one can write the following 
linear programming relaxation for the survivable network design problem, denoted by 
LP S ndp, with the function / being skew supermodular. 

minimize c e x e 

subject to x(S(S)) > f(S) 

< x e < 1 Vee£ 

It is not known whether there is a polynomial time separation oracle for a general skew 
supermodular function /. This linear program for the minimum Steiner network prob- 
lem, however, can be solved in polynomial time by using a maximum flow algorithm as a 
separation oracle. 

10.1.2 Characterization of Extreme Point Solutions 

For a subset S C V, the corresponding constraint x(S(S)) > f(S) defines a characteristic 
vector x(<5(<S*)) in ~^ E ^- the vector has an 1 corresponding to each edge e 6 5(S), and a 
otherwise. By the strong submodularity of the cut function of undirected graphs (see 
Section 2.3.1), we have 

x(6(X)) + x(S{Y)) > x(5(X n Y)) + x(5(X U Y)) and 

x(5(X)) + x(S(Y)) > x(5(X - Y)) + x{S(Y - X)), 

for any two subsets X and Y. When / is skew supermodular, it follows from standard 
uncrossing arguments, as in spanning trees (Chapter 4) and maximum matchings (Chap- 
ter 9), that an extreme point solution to LP sn d p is characterized by a laminar family of 
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tight constraints. Lemma 10.2.1 below follows from these uncrossing arguments and the 
Rank Lemma. 

Lemma 10.1.1 Let the requirement function f of LP sn d p be skew supermodular, and let x 
be an extreme point solution to LP sn( i p with < x e < 1 for every edge e 6 E. Then, there 
exists a laminar family C such that: 

(i) x(5(S)) = f{S) for each S € C. 

(ii) The vectors in {x(S(S)) : S G £} are linearly independent. 
(hi) \E\ = \C\. 



10.1.3 Iterative Algorithm 

The following is Jain's iterative rounding algorithm. 

Iterative Minimum Steiner Network Algorithm 

(i) Initialization F <— 0, /' <— /; 

(ii) While /' do 

(a) Find an optimal extreme point solution x to LP sn d p with cut requirement 
/' and remove every edge e with x e = 0. 

(b) If there exists an edge e with x e > 1/2, then add e to F and delete e from 
the graph. 

(c) For every S C V: update f(S) <- max{/(5) - d F (S),0}. 

(iii) Return H = (V,F). 



Fig. 10.1. Minimum Steiner Network Algorithm 



10.1.4 Correctness and Performance Guarantee 

Jain developed a token counting argument and proved an important theorem about the 
extreme point solutions to LP sn d p - 

Theorem 10.1.2 Suppose f is an integral skew supermodular function and x is an extreme 
point solution to LP sn< i p . Then there exists an edge e G E with x e > \. 

Assuming Theorem 10.1.2, the iterative algorithm will terminate, and it can be 
shown by an inductive argument that the returned solution is a 2-approximate solution 
(see Theorem 6.3.2 for a similar proof). We prove Theorem 10.1.2 by a counting argument, 
following the original proof of Jain, which will be useful later for the minimum bounded 
degree Steiner network problem. 
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Suppose for a contradiction that every edge e has < x e < |. By Lemma 10.2.1, 
there is a laminar family £ of tight constraints that defines x. We assign two tokens to 
each edge, one to each endpoint, for a total of 2\E\ tokens. Then we will redistribute the 
tokens so that each member in C receives at least 2 tokens and there are some tokens left. 
This would imply that \E\ > \L\, contradicting Lemma 10.2.1. 

Let S be a subset of vertices. Define its co-requirement coreq(S) := J2e^S(S)(h ~ x e)- 
We call a set S G C special if coreq(S) = \. Given a laminar family, it can be represented 
as a forest of trees if its sets are ordered by inclusion. Recall that in this forest, a set R G C 
is a child of S 6 C if S is the smallest set in C that contains R. We say an endpoint v is 
owned by S if S is the smallest set in C that contains v. The following lemma is useful to 
establish that a certain set is special. A proof can be found in Vazirani's book [161], and 
it is omitted here. 

Lemma 10.1.3 Suppose S has a children and owns (3 endpoints where a + /3 = 3. // all 
children of S are special, then S is special. 

The redistribution of tokens is by an inductive argument based on the following 
lemma, which would yield the contradiction that \E\ > \L\. 

Lemma 10.1.4 For any rooted subtree of the forest C with root S, the tokens assigned to 
vertices in S can be redistributed such that every member in the subtree gets at least two 
tokens, and the root S gets at least three tokens. Furthermore, S gets exactly three tokens 
only if S is special; otherwise S gets at least four tokens. 

Proof The proof is by induction. For the base case, consider a leaf node S in the laminar 
family. Since f{S) > 1 and x e < \ for all e, this implies that d(S) > 3 and thus S 
can collect three tokens. Furthermore, S collects exactly three tokens only if coreq(S) = 
d(S)/2 — x(5(S)). In this case, f(S) = 1 and d(S) = 3, and thus S is special. This verifies 
the base case. 

For the induction step, consider a non-leaf node S. For a child R of S, we say R has 
one excess token if R has three tokens, and R has at least two excess tokens if R has at 
least four tokens. By the induction hypothesis, each child has at least one excess token, 
and has exactly one excess token only if it is special. We will divide the cases by the 
number of children of S. 

(i) S has at least 4 children: Then S can collect 4 tokens by taking one excess token 
from each child. 

(ii) S has 3 children: If any child of S has two excess tokens or if S owns any endpoint, 
then S can collect 4 tokens. Otherwise, all three children of S are special and S 
does not own any enpdoint, but then S is special by Lemma 10.1.3, and so 3 tokens 
are enough for the induction hypothesis. 

(iii) S has 2 children: If both children have two excess tokens, then S can collect 4 
tokens. So assume that one child of S is special, but then it can be shown that S 
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must own at least one endpoint (see the exercises or Lemma 23.20 of [161]). If both 
children of S are special, then S is special by Lemma 10.1.3, and so three tokens 
are enough for the induction hypothesis. Otherwise S can collect 4 tokens, 
(iv) S has one child R: Since and x(<5(i?)) are linearly independent, S must 

owns at least one endpoint. As both f(S) and f(R) are integers and there is no 
edge of integral value, this actually implies that S cannot own exactly one endpoint, 
and thus S owns at least two endpoints. If R is not special, then S can collect 4 
tokens; otherwise, S is special by Lemma 10.1.3 and so 3 tokens are enough for the 
induction hypothesis. 

□ 

The extra tokens in the roots of the laminar family give us the desired contradiction. 
This proves Theorem 10.1.2, and thus the 2-approximation algorithm for the survivable 
network design problem follows. 

10.2 Connection to the Traveling Salesman Problem 

In this section we formulate a generalization of the survivable network design problem and 
the traveling salesman problem. Then we present a proof that any extreme point solution 
of this generalization has an edge e with x e > |, and in some cases, x e = 1. This will 
generalize Jain's result and a result of Boyd and Pulleyblank on the traveling salesman 
problem. The new proof uses the fractional token technique, and does not need the notion 
of special sets used in the previous section to prove Jain's result. 

10.2.1 Linear Programming Relaxation 

The following is a linear programming relaxation which models both the survivable network 
design problem and the traveling salesman problem, denoted by LPf, where / is a skew 
supermodular function. It is similar to the linear programming relaxation for the minimum 
bounded degree Steiner network problem in the next section. 

minimize c e x e 

e<=E 

subject to x(S(S)) > f(S) VSCV 

x(6{v)) = f(v) \/v G W 

< x e < 1 Vee£ 

For the survivable network design problem we set f(S) = max u€ s,v^s { r uv} for each 
subset S C V and set W = 0. For the traveling salesman problem we set f(S) = 2 for 
each S CV, f{V) = and W = V. 
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10.2.2 Characterization of Extreme Point Solutions 

The following characterization follows from a straightforward application of the uncrossing 
technique using the fact that / is skew-supermodular and the same characterization of 
extreme point solutions as in Lemma 10.2.1 holds for LPf to give the following lemma. 

Lemma 10.2.1 Let the requirement function f of LPf be skew supermodular, and let x be 
an extreme point solution to LPf with < x e < 1 for every edge e £ E. Then, there exists 
a laminar family C such that: 

(i) x(S(S)) = f(S) for each S € £. 

(ii) The vectors in {x{S(S)) : S 6 C} are linearly independent. 

(iii) \E\ = \C\. 



10.2.3 Existence of Edges with Large Fractional Value 

Boyd and Pulleyblank [22] proved that there is an edge e with x e = 1 in any extreme point 
solution to the traveling salesman problem. The following theorem provides a common 
generalization to their result and Jain's result (Theorem 10.1.2). 

Theorem 10.2.2 Let f be an integral skew supermodular function, and x be an extreme 
point solution to LPf with x e > for all e. Then there exists an e £ E with x e > |. 
Moreover, if f(S) is an even integer for each subset SCV then there exists an edge e with 

X e = 1. 

Proof [First part]: We first prove that x e > \ for some edge e 6 E in any extreme point 
solution x to LPf. Suppose for a contradiction that < x e < \ for each e € E. Then we 
will show that \E\ > \C\, contradicting Lemma 10.2.1. The proof is by a fractional token 
counting argument. We give one token to each edge in E, and then we will reassign the 
tokens such that we can collect one token for each member in C and still have extra tokens 
left, giving us the contradiction that \E\ > \C\. Each edge e = uv is given one token which 
is reassigned as follows. 

(i) (Rule 1) Let S £ Che the smallest set containing u and R £ Che the smallest set 
containing v. Then e gives x e tokens each to S and R. 

(ii) (Rule 2) Let T he the smallest set containing both u and v. Then e gives 1 — 2x e 
tokens to T. 

We now show that each set S in C receives at least one token. Let S he any set with 
children R±, . . . , R^ where k > (if S does not have any children then k = 0). We have 
the following equalities. 

x(S(S)) = f(S) 
x(6(Ri)) = f(Ri) Vl<i<k 



163 



Subtracting we obtain, 

k k 

x(5(S)) -Y,x(6(Ri)) = f(S) - J2 f(Ri)- (10-1) 

i=\ i=l 
We divide the edges involved in the left hand side of (10.1) into three types, where 

A = {e : |en (U^)| = 0, \ef)S\ = 1} 
B = {e:\en(UiRi)\ = l,\enS\ = 2} 
C = {e: |en(Uii?i)| = 2, lenS 1 ! = 2}. 

Then (10.1) can be rewritten as: 




Fig. 10.2. Example for expression x(S(S)) — J2i=i x ($(Ri)) with k = 2 children. The dashed edges 
cancel out in the expression. Edge-sets A, B, C shown with their respective coefficients. 



:{A) _ x{B ) - 2x(C) = f(S) - f(Ri)- 



(10.2) 



Observe that A U B U C / 0; otherwise the characteristic vectors x(^(S)), x(5(i?i)), 
. . . , x((5(i?fc)) are linearly dependent. For each edge e £ A, S receives x e tokens from e 
by Rule 1. For each edge e 6 B, S receives 1 — x e tokens from e by Rule 1 and Rule 2. 
For each edge e G C, S receives 1 — 2x e tokens from e by Rule 2. Hence, the total tokens 
received by S is exactly 

J> e + ^(1-X e ) + J>-2* e ) 

eeA e£B eeC 

= x(A) + \B\ - x(B) + \C\ - 2x(C) 

k 

= \B\ + \C\ + f(S)-J2f(Ri), 

i=l 
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where the last equality follows from (10.2). Since / is integral and the above expression 
is non-zero (since A U B U C / 0), the right hand side is at least one, and thus every set 
S G C receives at least one token in the reassignment. 

It remains to show that there are some unassigned tokens, which would imply the 
contradiction that \E\ > \C\. Let R be any maximal set in C. Consider any edge e G 5(R). 
The fraction of the token by Rule 2 for edge e is unassigned, as there is no set T G C with 
\T f] e\ = 2, gives us the desired contradiction. This proves the first part of the theorem. 

[Second part]: The proof of the second part is almost identical to the proof of the first 
part, except that we use scaled token assignment rules. Each edge e = uv is given one 
token which it reassigns as follows. 

(i) (Rule 1') Let S be the smallest set containing u and R be the smallest set containing 
v in C. Then e gives x e /2 tokens each to S and R. 

(ii) (Rule 2') Let T be the smallest set containing both u and v. Then e gives 1 — x e 
tokens to T. 

We now show that each set in C receives at least one token. Let S be any set with children 
Ri,...,Rk where k > 0. We have the following equalities. 

x(6(S)) = f(S) 
x(S(Ri)) = f(R) \/\<i<k 

Dividing by two and subtracting we obtain, 

*(*(£)) - *Wm _ f(S) - f(Ri) 

2 2 
The edges involved in the left hand side are divided into types A,B,C exactly as in the 
first part. Then the above equation becomes 

<A) - xjB) _ _ /(g) -EiW) 
2 X[U) ~ 2 

Observe that iUBUC ^ 0; otherwise there is a linear dependence among the constraints 
for S and its children. Also, S receives tokens for each edge e G A by Rule 1', 1 — 
tokens for each edge e G B by Rule 1' and Rule 2', and 1 — x e tokens for each edge e G C 
by Rule 2'. Hence, the total tokens received by S is exactly 

eeA eeB eeC 

_ ^) + | B |_^ + | C |_ l(c) 

| B | , | C | , f(S)-T.,f(Ri) 

Since A U B U C / and / is an even- valued function, this is a positive integer. Thus 
every set S G C receives at least one token in the reassignment. 

Now, we show that there are some unassigned tokens showing the strict inequality 
\C\ < \E\. Let R be any maximal set C. Then, consider any edge e G 5{R). Then, token 
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by Rule 2' for edge e is unassigned as there is no set T £ £ with \TD e| = 2. This gives us 
the desired contradiction. □ 



10.3 Minimum Bounded Degree Steiner Networks 

In the minimum bounded degree Steiner network problem, given an undirected graph 
G = (V, E) with edge costs, connectivity requirements r uv for each pair u, v G V and 
degree bounds b v for each vertex v G V, the task is to find a subgraph H of G of minimum 
cost such that there are r uv edge disjoint paths between u and v for each u, v G V and 
degree of vertex v in H is at most b v . In general, the minimum bounded degree Steiner 
network problem does not admit any polynomial factor approximation algorithm, since 
the minimum cost Hamiltonian cycle problem is a special case. Instead, we will present 
bicriteria approximation algorithms, that both violate the degree bounds as well as delivers 
a suboptimal cost solution. In this section, we will first present a (2, 2i^+3)-approximation 
algorithm for the minimum bounded degree Steiner network problem, where B v denotes 
the degree upper bound on node v. By this, we mean that the algorithm outputs a Steiner 
network whose cost is at most twice that of the objective value of the linear programming 
relaxation, and whose degree at any node v is at most 2B V + 3. In the next section, we 
will present an algorithm with only an additive violation on the degree bounds. 



10.3.1 Linear Programming Relaxation 

The linear programming formulation for the minimum bounded degree Steiner network 
problem, denoted by LPbdsn, is a straightforward generalization of LP sn d p for the survivable 
network design problem, with f(S) = max u€ s,v<£S { r uv} for each subset S C V . Notice 
that the degree constraints are only present on a subset W C V of vertices. 



minimize y c e x e 

subject to x(5(S)) > f(S), \/S<ZV 

x(5(v)) < B v , Vu G W 

< x e < 1 VeG-E 



10.3.2 Characterization of Extreme Point Solutions 

As the degree constraints are defined only on single vertices, the same uncrossing technique 
(as in Lemma 10.2.1) can be applied to show that an optimal extreme point solution is 
characterized by a laminar family of tight constraints. 
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Lemma 10.3.1 Let the requirement function of LPbdsn be skew supermodular, and let x 
be an extreme point solution to LPbdsn with < x e < 1 for every edge e G E. Then, there 
exists a laminar family C of tight sets such that C partitions into a set of singletons £ for 
the degree constraints, and the remaining sets C" = £ — £' for the connectivity constraints, 
such that: 

(i) x(S(v)) = B V >0 for all {v} G C and x(5{S)) = f{S) > 1 for all S G C" . 

(ii) The vectors in {x(8(S)) : S G £} are linearly independent. 

(iii) \E\ = \C\. 

10.3.3 Iterative Algorithm 

The iterative algorithm is similar to that for the minimum Steiner network problem, with 
a new relaxation step where we remove degree constraints of vertices with low degree. 

Iterative Minimum Bounded Degree Steiner Network Algorithm 

(i) Initialization F <— ®, f <— f. 

(ii) While /' do 

(a) Find an optimal extreme point solution x with cut requirement /' and 
remove every edge e with x e = 0. 

(b) (Relaxation): If there exists a vertex v G W with degree at most 4, then 
remove v from W. 

(c) (Rounding): If there exists an edge e = (u, v) with x e > 1/2, then add e 
to F and delete e from the graph and decrease B u and B v by 1/2. 

(d) For every S <^V: update f'(S) <- max{/(5) - d F (S),0}. 

(iii) Return H = (V, F). 



Fig. 10.3. Minimum Bounded Degree Steiner Network Algorithm 



10.3.4 Correctness and Performance Guarantee 

The performance guarantee follows by an inductive argument as in Lemma 6.3.2, assuming 
the algorithm terminates. We will prove the following lemma showing that the algorithm 
will terminate. 

Lemma 10.3.2 Let x be an extreme point solution to LPbdsn with x e > for every edge 
e, and W be the set of vertices with degree constraints. Then either one of the following 
holds: 

(i) There exists an edge e with x e > ^. 

(ii) There exists a vertex v G W with degree d(v) < 4. 
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In the following we use a token counting argument to prove Lemma 10.3.2. The 
counting argument is similar to that of Theorem 10.1.2. Each edge is assigned two tokens, 
for a total of 2\E\ tokens. For each edge e, one token is assigned to each endpoint. We 
shall show that if none of the steps in the algorithm can be applied, then each set in C" 
and each degree constraint in £' can collect two tokens, and there are some tokens left. 
This would imply \E\ > \C\, contradicting that x is an extreme point solution. 

We prove the same statement as in Lemma 10.1.4 to get the contradiction. The 
counting argument is similar and we only highlight the differences here. By Step 2(b), we 
may assume that d(v) > 5 for each v £ W, and hence each degree constraint has three 
extra tokens. The case when the node S £ C does not own a vertex in W is exactly the 
same as in Lemma 10.1.4. Henceforth we consider a node S £ C that owns a vertex v £ W. 
If S has another child R, then S can collect three tokens from v and at least one token 
from R, as desired. Otherwise, by linear independence of x(<5(5')) and x(<5(v)), S owns at 
least one endpoint and thus S can collect three tokens from v and at least one more token 
from the endpoints it owns as required. This completes the proof of Lemma 10.3.2. 



10.4 An Additive Approximation Algorithm 

In this section we show how to achieve additive guarantee on the degree bounds that 
depend only on the maximum connectivity requirement, denoted by r max = max UyV {r uv }. 

Theorem 10.4.1 There is a polynomial time (2, -B„ + 6r max + 3)- approximation algorithm 
for the minimum bounded degree Steiner network problem, which returns a solution with 
cost at most twice the optimum while the degree of each vertex v is at most B v + 6r max + 3. 

For minimum bounded degree Steiner trees, since the maximum connectivity require- 
ment is one, this algorithm will output a solution that violates the degree bounds only by 
an additive constant, and of cost within twice the optimal. 



10-4-1 Iterative Algorithm 

The iterative algorithm uses the same linear programming relaxation LPmsu as in the 
previous section. The difference is that we only pick an edge e with x e > \ when both 
endpoints have "low" degrees. 

10.4-2 Correctness and Performance Guarantee 

First we show that the degree of any vertex v in the returned solution H is at most 
B v + Qr max + 3, assuming that the algorithm terminates. We define the set Wh of vertices 
(defined in Step (ii) (a) of the algorithm) with fractional degree at least 6r max as high 
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Additive Approximation for Minimum Bounded Degree Steiner Network 

(i) Initialization F <— 0, /' <— f . 

(ii) While /' do 

(a) Find an optimal extreme point solution x to LP^sn satisfying /' and re- 
move every edge e with x e = 0. Set Wh = {v G W | Y,eeS(v) x e > 6r maz } 
and = E e e<5(» for v £ W. 

(b) If there exists an edge e = (it, u) with x e = 1, then add e to F and remove 
e from G and decrease S u and .B„ by 1. 

(c) (Relaxation): If there exists a vertex v £ W with degree at most 4, then 
remove v from W. 

(d) (Rounding): If there exists an edge e = (u, v) with x e > \ and u, v ^ IV/j, 
then add e to F and remove e from G and decrease B u and by x e . 

(e) For every S CV: f'(S) <- max{/(5) - d F (5),0}. 

(iii) Return if = (V,F). 



Fig. 10.4. Additive Approximation for Minimum Bounded Degree Steiner Network 

degree vertices. Observe that the fractional degree of each vertex is non-increasing during 
the algorithm, since we reset the degree upper bound in Step (ii) (a) in each iteration. 
Consider an edge e with v as an endpoint. When v £ Wh, e is picked only if x e = 1 in 
Step (ii) (b) of the algorithm. Hence, while v 6 W^, at most B v — 6r max edges incident 
at v are added to H. While v € W \ Wh, e is picked only if x e > \ in Step (ii) (d) of 
the algorithm. Hence, while v 6 W \ Wh, strictly less than 12r max edges incident at v are 
added to H. Finally, by Step (ii) (c) of the algorithm, a degree constraint is removed only if 
v is incident to at most four edges, where possibly all of them are added to H . Therefore, 
the degree of v in H is strictly less than (B v - 6f max ) + I2f max + 4 = B v + 6f m ax + 4. 
As B v is an integer, the degree of v in H is at most B v + 6f m ax + 3. Moreover, since we 
always included edges with value at least half in F in each iteration, by induction, the cost 
of the final solution is at most twice the optimal objective value of LP^ sn . This proves 
Theorem 10.4.1, assuming that the algorithm terminates. 

The following lemma proves that the algorithm will always terminate. With the same 
characterization as in Lemma 10.3.1, we use a more careful counting argument to prove 
stronger properties of the extreme point solutions to LP^ sn than those in Lemma 10.3.2. 

Lemma 10.4.2 Let x be an extreme point solution to LP^n with x e > for all e, and W 
be the set of vertices with degree constraints, and Wh = {v & W \ X]ee<5(» Xe — ^ r max}- 
Then at least one of the following holds. 

(i) There exists an edge e with x e = 1. 

(ii) There exists an edge e = {u,v} with x e > 1/2 and u, v ^ Wh- 

(iii) There exists a vertex v G W with degree at most 4. 
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We will prove Lemma 10.4.2 by a token counting argument as in Lemma 10.3.2. 
Suppose for contradiction that none of the conditions in Lemma 10.4.2 holds. Then each 
edge e has x e < 1, and each edge e with \ < x e < 1 (we call such an edge a heavy edge) 
must have at least one endpoint in Wh, and each vertex in W must have degree at least 
five. We give two tokens to each edge for a total of 2\E\ tokens. Then, the tokens will be 
reassigned so that each member of C" gets at least two tokens, each vertex in £ gets at 
least two tokens, and there are still some excess tokens left. This will imply that \E\ > \C\, 
contradicting Lemma 10.3.1. 

The main difference from Lemma 10.3.2 is the existence of heavy edges (with an 
endpoint in Wh) which our algorithm is not allowed to pick. Since there exist heavy edges, 
a set S £ C may only have two edges in S(S), and hence S may not be able to collect 
three tokens as in Lemma 10.1.4. To get around this, we use a different token assignment 
scheme and revise the definition of co-requirement, so that a similar induction hypothesis 
would work. 

Token assignment scheme: If e = uv is a heavy edge with u 6 Wh and v ^ W, then v 
gets two tokens from e and u gets zero token. For every other edge e, one token is assigned 
to each endpoint of e. 

Co-requirement: We revise the definition of co-requirement for the presence of heavy 
edges: 

coreq(S)= Yl (1/2 - s e ) + Yl i 1 '^)- 

eeS(S), x e <l/2 eeS(S), x e >l/2 

It is useful to note that this definition reduces to the original definition of co-requirement, 
if every edge e with x e > \ is thought of as two parallel edges, each aiming to achieve a 
value of \ and each has fractional value : summing \ — ^f over both edges gives 1 — x e . 
We say a set is special if coreq(S) = \ as in the proof of Jain's theorem. 

After this initial assignment, each vertex in V \ Wh receives at least as many tokens 
as their degree. Moreover, each vertex in W \ Wh receive at least five tokens, as their 
degree is at least five. Note that a vertex v £ Wh might not have any tokens if all the 
edges incident at it are heavy edges. However, by exploiting the fact that f(S) < r max , we 
will show that vertices in Wh can get back enough tokens. We prove the following lemma 
which shows that the tokens can be reassigned in a similar way as in Lemma 10.1.4. 

Lemma 10.4.3 For any subtree of C rooted at S, we can reassign tokens such that each node 
in the subtree gets at least two tokens and the root S gets at least three tokens. Moreover, 
the root S gets exactly three tokens only if S is special; otherwise it gets at least four tokens. 

The following is a key claim showing if S owns a vertex in Wh, then there are enough 
tokens for S and the vertices in Wh that it owns. 

Claim 10.4.4 Suppose S owns w > 1 vertices in Wh- Then the number of excess tokens 
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from the children of S, plus the number of tokens owned by S, plus the number of tokens 
left with vertices in Wh fl5 is at least 2w + 4. 

Proof Suppose S has c children. As each child has at least one excess token by the induction 
hypothesis, if c > 6w then we have 6w tokens which is at least 2w + 4. Henceforth we 
assume that c < Qw. Let B := J2 v eW h x ($( v ))' where the sum is over all vertices v 6 Wh 
owned by S. Note that B > Qwr max by the definition of Wh- For a child R of S, as 
x(S(R)) = f(R) < r max , at most r max units of B are contributed by the edges in 5(R). 
Similarly, at most r max units of B are contributed by the edges in 5(S). Hence, at least 
r m ax(Qw — c — 1) units of B are from the edges with both endpoints owned by S. Since 
there is no edge e with x e = 1, there are at least r max (6w — c — 1) + 1 such edges. Let 
e = uv be such an edge with v 6 Wh owned by S. If u G W, then both u and v get one 
token from e in the initial assignment. If u ^ W, then u gets two tokens from e in the 
initial assignment, but these two tokens are owned by S. Hence, the number of tokens 
owned by S plus the number of tokens left with vertices in Wh owned by S is at least 
Tmaxi&w — c — 1) + 1. Furthermore, S can also collect one excess token from each child. 
So, the total number of tokens that S can collect is at least r max (6w — c — 1) +c + 1, which 
is a decreasing function of c. As c < 6w, the number of tokens is minimized at c = 6w — 1, 
which is at least 2w + 4, proving the claim. □ 

We now proceed by induction on the height of the subtree to prove Lemma 10.4.3. 

Base Case: S is a leaf node. Claim 10.4.4 implies that S can collect enough tokens if it 
owns vertices in Wh- Hence assume S D Wh = 0- First consider the case when S H W / 0. 
Any vertex v 6 W \ Wh has at least five tokens, and thus has three excess tokens. If S 
owns two such vertices or S owns another endpoint, then S gets at least four tokens as 
required. Otherwise, we have x(d( v )) = x($(S)), contradicting Lemma 10.3.1. 

Henceforth we consider the case when S (1 W = 0. Then S can get at least \6(S)\ = 
d(S) tokens from the vertices owned by S. Note that d(S) > 2, as x(5(S)) is an integer 
and there is no edge e with x e = 1. If d(S) > 4, then S gets at least four tokens. If 
d(S) = 3 and d(S) contains a heavy edge, then S can get four tokens from the vertices it 
owns. If it does not contain a heavy edge, then S receives three tokens and coreq(S) = \, 
for which three tokens are enough. If d(S) = 2, then at least one edge is a heavy edge. If 
both edges are heavy then S can get four tokens; otherwise if only one edge is heavy then 
coreq(S) = \ and so three tokens are enough. 

Induction Step: If S owns any vertex in Wh, then we are done by Claim 10.4.4. Thus we 
can assume that S does not own any vertex in Wh- By the induction hypothesis, any child 
of S has at least one excess token. So, if S owns a vertex in W \ Wh, then S can collect 
at least four tokens. Hence, we can assume that S does not own any vertex in W, and 
the proof of these cases are almost identical to that in Lemma 10.1.4 with the definition 
of co-requirement revised; the details are omitted. 
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10.4-3 Steiner Forests 



In the special cases of Steiner trees and Steiner forests, the connectivity requirement func- 
tion is a {0, l}-function and hence r max = 1. Theorem 10.4.1 thus implies that there is a 
(2, B v + 9)-approximation algorithm for this problem, but it is possible to obtain a better 
bound. 

Theorem 10.4.5 There is a polynomial time (2, B v + 3)- approximation algorithm for the 
minimum bounded degree Steiner network problem when r max = 1. 

The iterative algorithm is similar where the improvement comes from the following 
fact [111, 153]: there is a heavy edge between two vertices without degree constraints. The 
proof is by a more involved counting argument that uses a different induction hypothesis. 

Notes 

The algorithm for the survivable network design problem is due to Jain [91], who introduced 
the iterative rounding method in the design of approximation algorithms. Subsequently the 
iterative rounding method has been applied to other network design problems, including 
directed network design [120, 67, 96], element-connectivity network design and vertex- 
connectivity network design [54, 34]. The existence of an 1-edge in the traveling salesman 
problem is due to Boyd and Pulleyblank [22], and the proof presented in this chapter is from 
the work of Nagarajan, Ravi and Singh [127]. The algorithm for the minimum bounded 
degree Steiner network problem is by Lau, Naor, Salavatipour and Singh [110], who first 
used the iterative relaxation idea in degree-bounded network design problems. The subse- 
quent work with additive violation on the degree bounds is by Lau and Singh [111]. Louis 
and Vishnoi [114] improved the bicriteria bound to obtain a (2,25 + 2)-approximation. 
The weighted degree constrained network design problem were considered by Pukunaga 
and Nagamochi [61] and Nutov [132]. 

Exercises 

10.1 Prove Lemma 10.1.3. 

10.2 Show the following statement in the proof of Lemma 10.1.4: If set S has two 
children, one of which has a co-requirement of 1/2, then it must own at least one 
endpoint. 

10.3 Give a 2-approximation algorithm for the traveling salesman problem, using the 
fact there is always an edge e with x e = 1 in the linear programming relaxation 

10.4 Show that the result of Boyd and Pulleyblank on the existence of a 1-edge in the 
subtour relaxation for the TSP can be derived as a corollary of the result of Jain on 



172 



the existence of a |-edge for the SNDP. In particular show that an extreme point 
for the former scaled down by a factor of two is feasible and is an extreme point 
for the spanning tree case of SNDP to derive the result. 

10.5 Consider the subtour elimination LP for asymmetric TSP [84]. Find the largest 
value p such that there is always some arc variable with x a > p- 

10.6 Design a bicriteria approximation algorithm for the minimum bounded degree 
strongly fe-arc-connected subgraph problem in directed graphs. 

(a) Write a "cut-cover" linear programming relaxation for the problem. Show 
that the connectivity requirement function for this problem is a crossing 
supermodular function. 

(b) Use the uncrossing technique to show that an extreme point solution is 
defined by a cross-free family of tight constraints. 

(c) (Gabow [67]) Prove that there is an arc with value at least | in any extreme 
point solution when there are no degree constraints in the problem. 

(d) Apply the iterative relaxation step to obtain a bicriteria approximation al- 
gorithm for the problem. 

10.7 Let P be the Peterson graph, and let w\, W2,wz be the neighbors of a vertex v 
in P. Take three copies of P — v and three new vertices vi,V2,vs, and attach 
them as follows: for 1 < j < 3, add edges from Vj to each Wj in the three copies of 
P — v. Prove that the resulting graph is a 3-regular 3-edge-connected graph with no 
Hamiltonian cycle. By setting B v = 1 for each vertex and connectivity requirement 
r uv = 1 for each pair of vertices, show that this is an integrality gap example in 
which there is a feasible solution to LPmsu but there is no Steiner network with 
degree at most 2B V (or B v + 1) for each vertex. 

10.8 Show that the following example gives an integrality gap of at least an additive 
n/4 for the degree violation for LPmsu- The input is a complete bipartite graph 
B = (X, Y, E) where X = {x\, X2} and Y = {yi, . . . , y n }. We set the connectivity 
requirements between and r/j to be 1 for all i,j, between x\ and X2 to be |, and 
otherwise. 

10.9 Can you use the fractional token technique in Section 10.2 to obtain the results for 
the minimum bounded degree Steiner network problem presented in this chapter? 
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11 

Constrained Optimization Problems 



In previous chapters, we have used the iterative relaxation method to obtain approximation 
algorithms for degree bounded network design problems. In this chapter, we illustrate that 
similar techniques can be applied to other constrained optimization problems. In the first 
part we study the partial vertex cover problem, and show an iterative 2-approximation 
algorithm for the problem. In the second part we study the multicriteria spanning tree 
problem, and present a polynomial time approximation scheme for the problem. 



11.1 Vertex Cover 

We first give a simple iterative 2-approximation algorithm for the vertex cover problem, 
and then show that it can be extended to the partial vertex cover problem. 

Given a graph G = (V, E) and a cost function c on vertices, the goal in the vertex 
cover problem is to find a set of vertices with minimum cost which covers every edge, i.e. 
for every edge at least one endpoint is in the vertex cover. In Chapter 3 we showed that the 
vertex cover problem in bipartite graphs is polynomial time solvable, and gave an iterative 
algorithm for finding the minimum cost vertex cover. In general graphs, the vertex cover 
problem is NP-hard. Nemhauser and Trotter [129] gave a 2-approximation for the problem. 
Indeed they prove a stronger property of half-integrality of the natural linear programming 
relaxation. We prove this result and its extensions to the partial vertex cover problem in 
the next section. 



11.1.1 Linear Programming Relaxation 

The linear programming relaxation, denoted by LP VC , is the same as in Section 3.4 for the 
vertex cover problem in bipartite graphs. 
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minimize 2 c v %v 

vev 

subject to x u + x v > 1 Ve = {u,v}£E 

x v > Vt) € V 

We shall prove the following theorem of Nemhauser and Trotter using the simple 
characterization of the extreme point solutions to the linear program. 

Theorem 11.1.1 Let x be an extreme optimal solution to LP VC . Then x v 6 {0, |, 1} for 

each v £ V. 

Theorem 11.1.1 implies Theorem 11.1.2 as a corollary 

Theorem 11.1.2 There exists a 2- approximation algorithm for the vertex cover problem. 

Proof Let x be the optimal extreme point solution to LP VC . Construct a vertex cover by 
picking each vertex v such that x v = ^ or x v = 1. Theorem 11.1.1 implies that it is a 
feasible vertex cover and it costs at most twice that of the fractional solution x. □ 



11.1.2 Characterization of Extreme Point Solutions 

We use the same characterization as in Lemma 3.4.3 for the vertex cover problem in 
bipartite graphs. For a set W C V, let x(W) denote the vector in M) v \; the vector has 
an 1 corresponding to each vertex v £ W, and otherwise. This vector is called the 
characteristic vector of W. 

Lemma 11.1.3 Given any extreme point x to LP VC with x v > for each v G V, there 
exists F C E such that 

(i) x u + x v = 1 for each e = {u, v} G F . 

(ii) The vectors in {x({u, v}) : {u,v} G F} are linearly independent. 
(hi) \V\ = \F\. 

11.1.3 Iterative Algorithm 

We now give a simple iterative rounding algorithm for obtaining a 2-approximation algo- 
rithm for the vertex cover problem. 
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(i) Initialization W <— ; 




(ii) Find an optimal extreme point solution 


X tO LPy C . 


(iii) While E ± do 




(a) For all vertices v £ V with x v = 


1 include v £ W and remove u from G. 


(b) For all vertices v £ V with x v = 


0, remove v from G. 


(c) For all vertices v £ V with Xi, = 


|, include v £ W and remove v from G. 


(iv) Return W. 





Fig. 11.1. Vertex Cover Algorithm 



11.1.4 Correctness and Performance Guarantee 

We now present the proof of Theorem 11.1.1 which also shows the correctness of the 
algorithm in Figure 11.1. 

Proof In Step (iii) (a), we select all vertices with x v = 1 and remove all such vertices. Then 
any vertex with x v = must now be an isolated vertex and hence can be removed. Thus 
we obtain a vertex cover instance on the remaining graph. We assume the remaining graph 
is connected; otherwise we apply the same argument to each connected component. From 
Lemma 11.1.3 there is a subset of edges F with linearly independent tight constraints. As 
\F\ = \ V\, it follows that F contains a cycle G. We will show that x v = \ for each vertex 
in G. First notice that G must be an odd cycle; otherwise the characteristic vectors in 
{x({u, v}) : {u, v} £ E(C)} are linearly dependent (the sum of the vectors for the "odd" 
edges is the same as that for the "even" edges in the cycle). As G is an odd cycle, the unique 
solution to these equations is x v = \ for each v £ C. Now, observing that x v + x u = 1 for 
any {u,v} £ F and x v = \ imply that x u = \, each vertex u reachable from the cycle G 
in F must also have x u = 5, proving Theorem 11.1.1. □ 



11.2 Partial Vertex Cover 

We now show how to extend the 2-approximation algorithm for the vertex cover problem 
to the more general partial vertex cover problem. Given a graph G = (V, E) and a cost 
function c on vertices and a bound L, the goal in the partial vertex cover problem is to find 
a set of vertices with minimum cost which covers at least L edges. The problem is NP-hard 
since it generalizes the vertex cover problem when L = \E\. We give a 2-approximation 
algorithm based on iterative rounding of a natural linear programming relaxation. 

Theorem 11.2.1 There is a 2-approximation algorithm for the partial vertex cover prob- 
lem. 
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11.2.1 Linear Programming Relaxation 



We proceed by first performing a pruning step where we guess the costliest vertex, say v 
in the optimal solution. We then obtained the pruned graph where we remove all vertices 
with cost more than v and include v in the solution. Since there are at most n = \V\ 
choices for the costliest vertex, we can consider all these choices and return the cheapest 
of the solutions returned by the algorithm on each of the pruned graphs. 

We now formulate a linear program for the pruned instance. The linear program, 
denoted by LP pvc , has a variable x v for each vertex v and a variable y e for each edge e G E. 
In an integral solution y e is set to one only if one of the endpoints is picked in the partial 
vertex cover. 



minimize 


/ C-v Xv 












vev 
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As we proceed with the iterative algorithm, we will work with a graph where edges could 
be of size one only. For example, when we have a variable with x v = 0, we will remove v 
from all edges containing v. Such edges will contain only one vertex but not two vertices. 



11.2.2 Characterization of Extreme Point Solutions 

We give a simple characterization of the extreme point solutions based on the Rank Lemma. 

Lemma 11.2.2 Given an extreme point solution x to LP pvc with < x v < 1 for all v G V 
and < y e < 1 for all e G E, there is a subset F C E of edges such that 

(i) Y^vae x v = Ve for each e G F. 

(ii) The constraints in {^2 v ^ e x v = y e : e G F} U {^2 ee EVv = L} are linearly indepen- 
dent. 

(hi) \F\ + 1 = \V\ + \E\. 



11.2.3 Iterative Algorithm 

The following is an iterative algorithm for the partial vertex cover problem. 
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Iterative Partial Vertex Cover Algorithm 

(i) Initialization W <— 0; 

(ii) While E ± do 

(a) Find an optimal extreme point solution (x, y) to LP pvc . If there is an edge 
e £ E with y e = 0, then remove e from G. If there is a vertex v G V with 
x v = 0, then remove v from G and from all the edges containing it, i.e, 
e <— e \ {f } for all e £ E. 

(b) If there is a vertex v € V with x v >\, then include v £ W and remove i> 
and all the edges incident at u from G. Update L<-L-|{e:e9t)}|. 

(c) If G contains a single vertex v, then include v £ W and remove v and all 
the edges incident at v from G. Update L <— L — \{e : e 3 v }|. 

(iii) Return W. 



Fig. 11.2. Partial Vertex Cover Algorithm 

11.2.4 Correctness and Performance Guarantee 

We now show that the algorithm in Figure 11.2 is a 2-approximation algorithm for the 
pruned instances, proving Theorem 11.2.1 for the correct guess of the costliest vertex in 
the optimal solution. The following lemma shows that the algorithm will terminate. 

Lemma 11.2.3 Let G be a graph with \V(G)\ > 2. Then at least one of the following must 
hold. 

(i) There exists a vertex v with x v 6 {0, 1}. 

(ii) There exists an edge e with y e = 0. 

(iii) There exists an edge e with y e = 1 and therefore x v > \ for some v 6 e. 

Proof Suppose for contradiction that none of the above condition holds. Then we have 
< x v < 1 for all vertices and < y e < 1 for all edges. From Lemma 11.2.2 there exists a 
subset of edges F such that |F| + 1 = \E\ + \V\. Since \F\ < \E\, this implies that \V\ < 1, 
a contradiction. □ 

Now, we prove that the iterative algorithm is a 2-approximation algorithm for the 
correct guess of the costliest vertex. 

Proof of Theorem 11.2.1: Whenever we pick a vertex with x v > \, a simple 
inductive argument shows that we pay a cost of at most twice the optimal fractional 
solution. Moreover whenever y e = 1 we must have x v > \ for some v 6 e as |e| < 2 for 
each edge e. For the last vertex picked in Step (ii) (c) , the cost of this vertex is at most the 
cost of the costliest vertex. Since the LP value of the costliest vertex was set to 1 in the 
preprocessing step, the cost of the last vertex picked is also charged to the LP solution. ■ 
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11.3 Multi-criteria Spanning Trees 



In the multi-criteria spanning tree problem, we are given a graph G = (V 7 , E) and cost 
functions Co, c\, . . . , c& on the edges and bounds L\, L2, ■ ■ ■ , L^ for each of the cost function 
Cj for 1 < i < k. The goal is to find a minimum co-cost tree which has Q-cost at most Lj. 

Ravi and Goemans [138] gave an algorithm for two cost functions cq and c\ which, 
given a positive e, returns a tree T with optimal Co-cost and C\(T) < (l + e)Li. The running 
time of the algorithm is polynomial for any fixed e. We present the following generalization 
of their result. 

Theorem 11.3.1 Given a graph G = (V, E) and cost functions Co, Ci, . . . , c& on the edges 
and bounds Li, L2, ■ ■ ■ , Lj- for each of the cost function except cq. Given any fixed e > 
there exists an algorithm which returns a tree of optimal CQ-cost and has Ci-cost at most 
(1 + e)Lj. The running time of the algorithm is polynomial for fixed k and e. 



11.3.1 Linear Programming Relaxation 

We formulate the following linear programming relaxation, denoted by LP mcst , which is 
a straightforward extension of the linear program for minimum spanning tree problem 
considered in Section 4.1. 



minimize J cn(e) x f 



J^co(e) 



subject to x(E(V)) = \V\ - 1, 

x(E(S)) < \S\-1, VScV 
s ^Ci(e)x e < Li, \/l<i<k 



e&E 

x e > Me£E 



11.3.2 Characterization of Extreme Point Solutions 

We now give a characterization of the extreme point solutions to LP mcs t. This follows 
directly from the Rank Lemma and the characterization of extreme point solutions of the 
spanning tree linear program considered in Section 4.1. 

Lemma 11.3.2 Let x be an extreme point solution to LP mcs t with x e > for each edge e. 
Then there exists a set J C {1, . . . , k} and a laminar family C such that: 

(i) YleeE c i( e ) x e = Li for each i 6 J and x(E(S)) = \S\ — 1 for each S G C. 

(ii) The vectors in {x(E(S)) : S G £} are linearly independent. 
(hi) |£| + |J| = |£|- 
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11.3.3 Iterative Algorithm 



The first step in the algorithm is a pruning step. Observe that no feasible solution can 
include an edge whose Q-cost is more than Lj. We extend this step further and guess all 
the edges in the optimal solution whose Q-cost is at most For any i there can be at 

most | such edges in the optimal solution. Thus the number of possibilities is at most 

k 

m e where m denotes the number of edges. There are k cost functions to try, and so the 

total possibilities is at most m e . After guessing these edges correctly, we throw away all 
other edges which have Cj-cost at least |_Lj and contract the guessed edges in the input 
graph. Clearly, the rest of the edges in the optimal solution form a spanning tree in the 
contracted graph, for the correct choice of the guessed edges. 

Now we have an instance where Cj(e) < |Lj for each e and i. We update the 
bound Li by subtracting the Q-costs of the guessed edges, and let L\ denote the residual 
bounds. We solve the linear program LP mcst with the updated bounds L^. Step (iii) of the 
algorithm can be interpreted as removing all the k constraints bounding the length under 
the cost functions c±, . . . , c^. Removing these constraints gives us the linear program for 
the spanning tree problem which is integral and its optimal solution is a minimum co-cost 
spanning tree. 

Algorithm for Multi-Criteria Spanning Tree 

(i) Guess all the edges in the optimal solution with Cj(e) > |Lj for some i € [k]. 
Include these edges in the solution and contract them. Delete all other edges from 
G with Cj(e) > |Lj. Update Lj for all i € [k]. 

(ii) Find an optimal extreme point solution x to LP mcst and remove every edge e 
with x e = 0. 

(iii) Pick any minimum co-cost tree in the support graph. 



Fig. 11.3. Multi-criteria Spanning Tree Algorithm 



11.3.4 Correctness and Performance Guarantee 

To prove Theorem 11.3.1, first we claim that the support of LP mcst on a graph with n 
vertices has at most n + k — 1 edges. In fact, from Lemma 11.3.2, we have \E\ = \C\ + \ J\. 
But \C\ < n — 1 since £ is a laminar family without singletons (see Proposition 4.1.7) and 
I J| < k proving the claim. By the results in Chapter 4, the Co-cost of the tree T returned 
by the algorithm is at most the co-cost of the LP-solution and hence is optimal for the 
correct guess of costly edges. Now we show that the Cj-cost returned by the algorithm is at 
most L\ + eLj. Note that any tree must contain n — 1 edges out of the n + k — 1 edges in 
the support graph. Hence, the maximum Cj-cost tree has cost at most k ■ j^Li = eLi more 
than that of the minimum Q-cost tree in this support graph. In turn, the minimum Q-cost 
tree has cost at most the Q-cost of the optimal fractional solution, which is at most L\ 
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by feasibility. Altogether, the maximum Cj-cost of the solution returned is no more than 
L\ + eLi. Adding the cost of the edges guessed in the first step we obtain that the tree 
returned by the algorithm has Cj-cost at most [L\ + eLi) + (Lj — L^) = (1 + e)Lj, proving 
Theorem 11.3.1. 



Notes 

The paper by Bshouty and Burroughs [24] was the first to give a 2-approximation algorithm 
for the partial vertex cover problem using LP-rounding. Subsequently other combinato- 
rial 2-approximation algorithms based on the local-ratio method [10] and the primal-dual 
algorithm [76] were proposed. 

Ravi and Goemans [138] presented a polynomial time approximation scheme (PTAS) 
for the bicriteria spanning tree problem via the Lagrangian relaxation method. The re- 
sult presented in this chapter for the multi-criteria spanning problem is from the work of 
Grandoni, Ravi and Singh [78]. 



Exercises 

11.1 Weighted Partial Vertex Cover: In an instance of weighted vertex cover problem, 
we are given a graph G = (V, E) with cost function c on vertices and a weight 
function w on edges and a weight target B. The goal is to find a vertex set V C V 
of minimum cost such that the edges covered by V' weight at least B. 

(a) Write a linear programming formulation for the partial vertex cover problem. 

(b) Extend the proof of Theorem 11.2.1 to obtain a 2-approximation for this 
problem. 

11.2 Partial Set Cover: In an instance of the set cover problem, we are given a ground 
set V, a collection of subsets S = {Si, S2, ■ ■ ■ , S m } of V with a cost function c on 
the subsets, a weight function w on the elements of V and a weight target B. The 
goal is to find a subset S' C S of minimum cost such that the elements covered by 
S' weight at least B. 

(a) Write a linear programming formulation for the partial set cover problem. 

(b) Extend the proof of the weighted partial vertex cover problem to obtain a 
/-approximation algorithm for the problem, where / denotes the maximum 
number of subsets in S containing a particular element in V. 

11.3 Prize Collecting Vertex Cover: In an instance of Prize Collecting Vertex Cover, 
we are given a graph G = (V, E) with cost function c on vertices and a penalty 
function w on edges. The cost of any solution defined by V C V is the cost of 
vertices in V' plus the penalty of the edges not covered by V'. The task is to find 
a subset V C V of minimum total cost. 
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(a) Write a linear programming formulation for the prize collecting vertex cover 
problem. 

(b) Give a simple iterative procedure to achieve a 2-approximation for the prob- 
lem. 

(c) Can you show that approximation factor of 2 needs to be paid only for 
the vertex cost and not for the edge penalties, i.e., the total cost of your 
approximate solution is at most the penalty cost for the edges paid by the 
LP solution plus twice the LP cost for the vertices. 

11.4 Multi-Criteria Matroid Basis: In this problem, we are given a matroid M = (V,I), 
cost functions Cj : V — > R for < i < k, bounds Lj for each 1 < i < k and the task 
is to the find the minimum co-cost basis of M such that Q-cost is at most Lj. 

(a) Write a linear programming relaxation for the multi-criteria matroid basis 
problem. 

(b) Extend the proof of Theorem 11.3.1 to give a polynomial time approximation 
scheme for the problem, with running time polynomial for fixed e and k. 
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12 

Cut Problems 



t 

In this chapter we present 2-approximation algorithms for three "cut" problems: the 
triangle cover problem, the feedback vertex set problem on bipartite tournaments, and the 
node multiway cut problem. All the algorithms are based on iterative rounding but require 
an additional step: as usual the algorithms will pick variables with large fractional values 
and compute a new optimal fractional solution iteratively, but unlike previous problems 
we do not show that an optimal extreme point solution must have a variable with large 
fractional value. Instead, when every variable in an optimal fractional solution has a small 
fractional value, we will use the complementary slackness conditions to show that there are 
some special structures which can be exploited to finish rounding the fractional solution. 
These algorithms do not use the properties of extreme point solutions, but we will need 
the complementary slackness conditions stated in Section 2.1.4. The results in this chapter 
illustrate an interesting variant and the flexibility of the iterative rounding method. 



12.1 Triangle Cover 

Given an undirected graph with weights on the edges, the triangle cover problem is to find 
a subset of edges F with minimum total weight that intersects all the triangles (3-cycles) 
of the graph, i.e. G — F is triangle free. 



12.1.1 Linear Programming Relaxation 

The following is a simple linear programming formulation for the triangle cover problem, 
denoted by LP tr i(G), in which x e is a variable for edge e and w e is the weight of edge e. 

t © Copyright 2011 by Lap Chi Lau, R. Ravi and Mohit Singh. Published 2011 by Cambridge 
University Press. 
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minimize w e x e 

subject to x e > 1 V triangle T 

eeT 

x e > Vee£ 



12.1.2 Iterative Algorithm 

The algorithm will first iteratively pick edges with large fractional value. When there is 
no edge with x e >\ and the graph is still not triangle-free, the algorithm will compute a 
bipartite subgraph H with at least half the total weight of the edges, and add all the edges 
not in H to the solution. 



Iterative Triangle Cover Algorithm 

(i) Initialization F <— 0. 

(ii) Compute an optimal fractional solution x to LP tr i(G). 

(iii) While there is an edge e with x e > \ do 

(a) For an edge e with x e > \, update F «— F U {e} and G 4— G — e. 

(b) Compute an optimal fractional solution x to LP tr i{G). 

(iv) If G is not triangle-free, then find a bipartite subgraph H with at least half the 
total weight, and update F<-FU (E(G) - E(H)). 

(v) Return F. 



Fig. 12.1. Triangle Cover Algorithm 



12.1.3 Correctness and Performance Guarantee 

The algorithm in Figure 12.1 can be implemented in polynomial time: there is a simple 
linear time algorithm to find a bipartite subgraph with at least half the total weight in 
Step (iv). In particular, consider the following greedy algorithm: process the nodes in an 
arbitrary order, and put them in one of two sides of the bipartition, greedily putting a 
node on the side so that at least half of the edges from it to the existing nodes crosses 
the bipartition. In the end, at least half of all the edges cross the bipartition, which is a 
2-approximation to the maximum- weight cut. 

To prove the approximation guarantee of the algorithm, the key step is to prove that 
Step (iv) gives a 2-approximate solution in the remaining graph, for which we will need 
the complementary slackness conditions. 
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Theorem 12.1.1 The algorithm in Figure 12.1 is a 2- approximation algorithm for the 
triangle cover problem. 

Proof The main difference from previous iterative algorithms is in the last step, where 
we add all the edges not in H to F. Notice that in this last step the algorithm does 
not consider the fractional values of the remaining edges, but only consider their weights. 
Suppose we could prove that 

J2 W e <2 W eXe, (12.1) 

eeE(G)-E(H) eeE(G) 

where the right hand side is twice the objective value in the remaining fractional solution 
after the while loop. Then we know that E(G) — E(H) is a 2-approximate solution for 
the triangle cover problem in the remaining graph, as the objective value is a lower bound 
on the optimal value. Since we only pick edges with fractional value at least \ in the 
loop, by a standard inductive argument as in previous chapters, we could show that the 
iterative algorithm in Figure 12.1 is an overall 2-approximation algorithm for the triangle 
cover problem. 

We now use the complementary slackness conditions to prove (12.1). The following 
is the dual program of LP tr i{G), in which there is one variable for each triangle T in G. 

maximize yx 
T 

subject to Vt < We Ve G E 

T:e<=T 

y T > V triangle T £ G 

We can assume without loss of generality that every edge e in G belongs to some triangle; 
otherwise we could delete e from G without changing any minimal solution to the triangle 
cover problem. Also, there is no edge with fractional value at least \ in Step (iv) of the 
algorithm because of the termination condition of the while loop. Since every edge e is 
in some triangle T, this implies that x e > for every edge e in Step (iv); otherwise some 
other edge in T must have a fractional value of at least \ in order to satisfy the constraint 
for T in LP tr i{G). As x is an optimal solution for LP tr i(G), by the primal complementary 
slackness conditions (see Section 2.1.4.1), in any dual optimal solution y we must have 

T:e£T 

for every edge e in G. Hence, the objective value of any dual optimal solution y is: 

T e£ET:e£T e&E 

where the first equality follows because every triangle is counted exactly thrice on the 
right hand side, and the second equality follows from the primal complementary slackness 
conditions. By the LP duality theorem (Theorem 2.1.9), the objective value of the primal 
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program is equal to that of the dual program. Therefore, in Step (iv) of the algorithm, the 
objective value for the primal solution is 

w e x e = ^2 VT = 3 Y We - 

As H computed in Step (iv) satisfies the property that YleeE(H) w e > \ SeG-E(G) w ei we 
have 

J2 W e< 2 We = 2 ^ WeXe ' 

e£E(G)~E(H) eeE(G) eeE(G) 

This implies (12.1), completing the proof of the theorem. □ 
A similar result holds for the directed triangle cover problem; see the exercises. 



12.2 Feedback Vertex Set on Bipartite Tournaments 

Given a directed graph D with weights on the vertices, the feedback vertex set problem is 
to find a subset of vertices S with minimum total weight that intersects all the directed 
cycles, i.e. D — S has no directed cycles. There is no known constant factor approximation 
algorithm for this problem. In this section we present a 2-approximation algorithm for the 
feedback set problem on a special class of directed graphs called bipartite tournaments. A 
bipartite tournament is obtained from an undirected complete bipartite graph by assigning 
exactly one direction to each edge, i.e. for each edge uv, exactly one of the arcs in {uv, vu} is 
present in the bipartite tournament. The feedback vertex set problem is easier in bipartite 
tournaments because of the following property. 

Proposition 12.2.1 A bipartite tournament has no directed cycle if and only if it has no 
directed cycle of length four. 

Proof Suppose there is a directed cycle of length 2k for k > 3, say X\,y\,X2, 2/2, • • • , X2k, U2k- 
Consider the orientation of the arc between x\ and y2- If it is oriented from y2 to x\, we 
have found a directed 4-cycle; else, we can shortcut the cycle to a 2{k — \) cycle and proceed 
by induction on k to complete the proof. □ 



12.2.1 Linear Programming Relaxation 

By Proposition 12.2.1 the problem of intersecting all directed cycles in a bipartite tourna- 
ment is equivalent to the problem of intersecting all directed 4-cycles. Thus we can write 
the following simple linear programming formulation for the feedback vertex set problem 
on bipartite tournaments, denoted by LP^ cyc {D)^ in which x v is the variable for vertex v 
and w v is the weight of vertex v. 
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minimize 2 w v x v 

subject to x v > 1 V directed cycle C of length 4 

vec 

x v > \/v £V 

12.2.2 Iterative Algorithm 

The algorithm will first iteratively pick vertices with large fractional value. When there is 
no vertex with x v >\, the algorithm will then add all the vertices with positive fractional 
value on one side of the bipartite graph to the solution. 

Iterative Feedback Vertex Set Algorithm on Bipartite Tournaments 

(i) Initialization S <— 0. 

(ii) Compute an optimal fractional solution to LP^ cyc {D\ 

(iii) While there is a vertex v with x v > \ do 

(a) For a vertex v with x v > |, update S <— S U {v } and D <— D — v. 

(b) Compute an optimal fractional solution x to LP^ cyc {D). 

(iv) Let the bipartition of D be X and Y, and let X + be the set of vertices in X with 
positive fractional value. 

(v) Return SUX+. 



Fig. 12.2. Feedback Vertex Set Algorithm on Bipartite Tournaments 



12.2.3 Correctness and Performance Guarantee 

First we prove that the solution returned by the algorithm is indeed a feedback vertex 
set. Every directed cycle C of length 4 has exactly two vertices in X and two vertices 
in Y, where X and Y are the bipartition of D. Let the vertices in C be {x±, yi, X2, J/2} 
where {x±,X2} Q X and {2/1,2/2} Q Y. After the loop, the fractional value of each vertex 
is smaller than \. So if C is still present after the loop, then at least one of X\ or X2 
must have positive fractional value to satisfy the constraint for C in LP^ cyc {D). Hence X + 
(as denned in Step (iv)) intersects all directed cycles of length 4 in the remaining graph. 
Therefore SU X + is a feedback vertex set in the original graph. To prove the performance 
guarantee, we will prove that X + is a 2-approximate solution in the remaining graph, for 
which we need the complementary slackness conditions. 

Theorem 12.2.2 The algorithm in Figure 12.2 is a 2- approximation algorithm for the 
feedback vertex set problem on bipartite tournaments. 
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Proof The main difference from standard iterative algorithms is in Step (iv), where we 
add all the vertices in X + to the solution regardless of their fractional values. Suppose we 
could prove that 

w v < 2 w v x v , (12-2) 
vex+ vev 

where the right hand side is twice the objective value in the remaining fractional solution 
after the while loop. Then we know that X + is a 2-approximate solution for the feedback 
vertex set problem in the remaining graph. As we only pick vertices with fractional value 
at least | in the loop, by an inductive argument as in previous chapters, we could show 
that the iterative algorithm in Figure 12.2 is an overall 2-approximation algorithm for the 
feedback vertex set problem in bipartite tournaments. 

We use the complementary slackness conditions to prove (12.2). The following is 
the dual program of LP^ cyc {D\ in which there is one variable for each directed cycle C of 
length 4 in D. 

maximize yc 

c 

subject to Vc < w v M v G V 

C:v&C 

yc > V directed cycle C of length 4 

As x is an optimal solution, by the primal complementary slackness conditions, in any 
optimal dual solution y, we must have 

^2 yc = w v 

C:veC 

for each vertex v G X + . Hence the objective value of any dual optimal solution y is 



„,„ _ 1 



c vex+ c-.vec v<=x+ 

where the first inequality follows because each directed 4-cycle intersects X + at most twice. 
By the LP duality theorem, we have 



^2 w v x v = ^2 vc > \ ^2 Wv - 

vev c «gx+ 

This proves (12.2), completing the proof of the theorem. □ 



12.3 Node Multiway Cut 

Given an undirected graph G with weights on the vertices and a subset T C V of terminal 
vertices, the node multiway cut problem is to find a subset S of vertices in V — T with 
minimum total weight that intersects every path with both endpoints in T, i.e. in G — S 
there is no path between any pair of vertices in T. 
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12.3.1 Linear Programming Relaxation 

The node multiway cut problem can be formulated by the following linear program, denoted 
by LP cut {G). In the following x v denotes the indicator variable for vertex v G V — T, w v 
denotes the weight of the vertex v G V — T, and V denotes the set of all paths between 
any two terminal vertices. 

> 1 VPeP 

> Vuel/-T 

There could be exponentially many constraints in LP cut (G), but this can still be solved by 
the ellipsoid algorithm in polynomial time, since we can use a shortest path algorithm to 
construct a separation oracle for this linear program. 

12.3.2 Iterative Algorithm 

The iterative algorithm is similar to the previous algorithms in this chapter. It will first 
iteratively pick vertices set to integral value. When there is no vertex with x v = 1, the 
algorithm will then add all the "boundary" vertices of the terminal vertices to the solution. 

Iterative Node Multiway Cut Algorithm 

(i) Initialization S <— 0. 

(ii) Compute an optimal fractional solution x to LP cut (G). 

(iii) While there is a vertex v with x v = 1 do 

(a) For a vertex v with x v = 1, update S <— S U {v } and G <— G — v. 

(b) Compute an optimal fractional solution x to LP cut (G). 

(iv) For each vertex v £ T, let Z v be the set of vertices which can be reached from v 
by paths of total fractional value zero, and let B v be the set of vertices that are 
adjacent to Z v , i.e. B v = {w G V — Z v \ 3u £ Z v with uw £ E}. 

(v) Return S U \J veT B v . 



Fig. 12.3. Node Multiway Cut Algorithm 



12.3.3 Correctness and Performance Guarantee 

Let B := U u ^tB u be the set of all boundary vertices of terminal vertices. Since each path 
between two terminal vertices has total fractional value at least one, it follows that B is a 



minimize w v x v 
veV-T 

subject to x v 
veP 
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node multiway cut in the remaining graph after the while loop; otherwise there would be a 
path between two terminals with total fractional value zero. To prove the approximation 
guarantee, we show that B is a 2-approximate solution in Step (iv), for which we will use 
(both) the complementary slackness conditions. 

Theorem 12.3.1 The algorithm in Figure 12.3 is a 2- approximation algorithm for the 
node multiway cut problem. 

Proof As in the previous algorithms in this chapter, the main difference from standard 
iterative algorithms is in the last step, where we add all the vertices in B to the solution 
regardless of their fractional values. Note that every vertex in B has positive fractional 
value. Suppose we could prove that 

Wv — ^ ^2 w vX v , (12-3) 

v<=B vEV-T 

which is twice the objective value in the remaining fractional solution after the while loop. 
Then we know that B is a 2-approximate solution for the node multiway cut problem in 
the remaining graph. As we only pick vertices with value 1 in the loop, by an inductive 
argument as in previous chapters, we could show that the iterative algorithm in Figure 12.1 
is an overall 2-approximation algorithm for the node multiway cut problem. 

We use the complementary slackness conditions to prove (12.3). The following is the 
dual program of LP cut (G), in which there is one variable for each path P of V . 

maximize ^ ^ yp 

P 

subject to VP ^ w v \/ v £ V — T 

P:veP 

y P > o ypev 

As x is an optimal solution, by the primal complementary slackness conditions, in any 
optimal dual solution y, we must have 

^2 yp = w v 

p-.veP 

for each vertex v G B. By the dual complementary slackness conditions, in any optimal 
primal solution x, we must have 

^2,x v = 1 

v£P 

for each path P € V with yp > 0. 

We now argue that each path P with yp > contains exactly two vertices in B. Let 
P be a path with yp > connecting two terminals s and t. Since there is no vertex v with 
x v = 1, B s and B t must be disjoint; otherwise there would be a path from s to t with total 
fractional value less than one. Hence P must contain one vertex in B s and one vertex in 
B t . Suppose to the contrary that P contains another vertex in B r . If r € {s, t}, then we 
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can shortcut the path P to use one fewer vertex in B and obtain a shorter path P' from s 
to t. However, since every vertex in B has a positive fractional value and YlveP x v = lhy 
the dual complementary slackness condition, this implies that Ylv£P' x v < 1, contradicting 
the fact that x is a feasible solution to LP cut {G). If r ^ {s,t}, then we can shortcut the 
path P to obtain a path P" from s to r, which is shorter than P since it avoids a vertex in 
B t . This implies that YlveP" x v < 1, as every vertex in B has a positive fractional value 
and J2 V £P x v = 1- Since P" is a path connecting two terminals s and r, this contradicts 
the fact that £ is a feasible solution to LP cut (G). Therefore, each path P with yp > 
contains exactly two vertices in B. Hence the objective value of any dual optimal solution 

y is 

P v&B P:v&P veB 

where the first equality holds because each path P with yp > uses exactly two vertices 
in B, and the second equality follows from the complementary slackness conditions. By 
the LP duality theorem, we have 



u>vx v = Y yp = \ Y Wv - 

veV-T P v£B 

This proves (12.3), completing the proof of the theorem. □ 



Notes 

The result for the triangle cover problem is from the work of Krivelevich [107]. The 
algorithm for the feedback vertex set problem on bipartite tournaments is due to van 
Zuylen [159], improving a 3.5-approximation algorithm by Cai, Deng and Zang [25] and 
an iterative rounding 3-approximation algorithm by Sasatte [146]. The result for the node 
mulitway cut problem is due to Garg, Vazirani and Yannakakis [71]. 



Exercises 

12.1 Design a ^-approximation algorithm for the maximum weighted bipartite subgraph 
problem. 

12.2 Consider the directed triangle cover problem: Given a directed graph with weights 
on the arcs, find a subset of arcs with minimum total weight that intersect all the 
directed triangle. Obtain a 2-approximation algorithm for this problem. 

12.3 Consider the following "complement" problem to the triangle cover problem: Given 
an undirected graph with weights on the edges, find a triangle-free subgraph with 
maximum total weight. Design a |-approximation algorithm for this problem. 
(Hint: use the 2-approximation algorithm for the triangle cover problem and the 
ideas therein.) 
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12.4 Generalize the result for the triangle cover problem to obtain a (k— l)-approximation 
algorithm for the /c-cycle cover problem when k is odd. What about the case when 
k is even? 

12.5 Consider the feedback vertex set problem on tournaments. A tournament is ob- 
tained from a complete undirected graph by assigning exactly one direction to each 
edge. 

(a) Prove that a tournament has a directed cycle if and only if it has a directed 
3-cycle. 

(b) Use part (a) to write a linear programming relaxation for the problem. 

(c) Obtain a 3-approximation algorithm for the problem using the above linear 
program. 

12.6 Consider the directed 4-cycle cover problem in bipartite directed graph (a directed 
graph with the underlying undirected graph bipartite): Given a bipartite directed 
graph with weights on the arcs, find a subset of arcs with minimum total weight 
to intersect all the directed 4-cycles. Design a 2-approximation algorithm for this 
problem. 

Can you also obtain a 2-approximation algorithm for the node- weighted case 
where the objective is to find a subset of nodes with minimum total weight to 
intersect all the directed 4-cycles? 

12.7 Consider the feedback arc set problem on bipartite tournaments, where the objec- 
tive is to find a subset of arcs with minimum total weight to intersect all the directed 
cycles. What is the best performance guarantee you can obtain in polynomial time? 

12.8 Work out the details of the separation oracle for solving LP cut . 

12.9 Show that there is an approximation-ratio preserving reduction from the minimum 
vertex cover problem to the node multiway cut problem. 

12.10 Prove that LP cut (G) is actually half-integral: there is always an optimal solution 
to LP cut (G) with x v G {0, |, 1} for every vertex v G V(G). 

12.11 Consider the multicut problem on trees: Given a tree T with weights on the edges 
and a set of I pairs of vertices {(sj, U) : 1 < i < I}, the objective is to find a subset 
of edges F with minimum total weight that intersect all the Sj-tj paths, i.e. there 
is no Sj-tj path in T \ F for 1 < i < I. 

(a) Write a linear programming relaxation for this problem. 

(b) Assume the tree T is rooted at a specific vertex r. An instance of the 
problem is called non-crossing, if for each pair (sj,£j), Sj is a descendant 
of ti in the rooted tree (T,r). Prove that the above linear programming 
relaxation always has integral optimal solutions for non-crossing instances. 

(c) Use part (b) to obtain a 2-approximation algorithm for the multicut problem 
on trees. (Hint: construct a non-crossing instance from a non-non-crossing 
instance.) 
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12.12 Can you obtain a 2-approximation algorithm for the multicut problem on trees by 
an iterative rounding algorithm? 
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13 

Iterative Relaxation: Early and Recent Examples 



t 

Even though we mentioned the paper by Jain [91] as the first explicit application 
of the iterative method to approximation algorithms, several earlier results can be reinter- 
preted in this light, which is what we set out to do in this chapter. We will first present 
a result by Beck and Fiala [14] on hypergraph discrepancy, whose proof is closest to other 
proofs in this book. Then we will present a result by Steinitz [156] on rearrangements of 
sums in a geometric setting, which is the earliest application that we know of. Then we 
will present an approximation algorithm by Skutella [152] for the single source unsplittable 
flow problem. Then we present the additive approximation algorithm for the bin packing 
problem by Karmarkar and Karp [93], which is still one of the most sophisticated uses of 
the iterative relaxation method. Finally, we sketch a recent application of the iterative 
method augmented with randomized rounding to the undirected Steiner tree problem [23] 
following the simplification due to Chakrabarty et al. [28]. 



13.1 A Discrepancy Theorem 

In this section we present the Beck-Fiala theorem from discrepancy theory using an iterative 
method. Given a hypergraph G = (V,E), a 2-coloring of the hypergraph is defined as an 
assignment ip : V — > { — 1, +1} on the vertices. The discrepancy of a hyperedge e is defined 
as disc x (e) = X^ee ip(v), and the discrepancy of the hypergraph G is defined as disc^{G) = 
max eg£(G) \{disc^p(e)}\. Beck and Fiala gave an upper bound on the discrepancy based on 
the maximum degree of the hypergraph (defined as maxi, |{e : v £ e}\). 

Theorem 13.1.1 Given a hypergraph G = (V,E) with maximum degree d, there is a 
coloring yj with discrepancy disc^p(G) < 2d — 1. 

t © Copyright 2011 by Lap Chi Lau, R. Ravi and Mohit Singh. Published 2011 by Cambridge 
University Press. 



194 



13.1.1 Linear Programming Relaxation 

Consider the following feasibility linear program LPdisc for the hypergraph 2-coloring prob- 
lem. There is a variable x v for each vertex v £ V, which is either 1 or —1 depending on 
whether ip(v) = 1 or ip(v) = —1. Initially we set B e = for each hyperedge e G E, as 
x v = for all v is a feasible solution to this linear program (even though it may not be an 
extreme point solution). 

x v = B e V e G E 

vde 

-l < x v < l V«eF 



13.1.2 Characterization of Extreme Point Solutions 

For a hyperedge e the corresponding constraint Ylvee Xr " = defines a characteristic 
vector x '■ B — > M} v \ with an 1 corresponding to each vertex v G e and otherwise. The 
following characterization is a direct application of the Rank Lemma. 

Lemma 13.1.2 Given any extreme point solution to LP<n sc with — 1 < x v < 1 for each 
v G V, there exists a subset F C E such that 

(i) Z^ee x v = B e for each e G F. 

(ii) The characteristic vectors for the constraints in F are linearly independent. 

(iii) \F\ = \V\. 

13.1.3 Iterative Algorithm 



The iterative algorithm is similar to that for the minimum bounded degree spanning tree 
problem. In each iteration we either fix the variables with value +1,-1 or remove the 
constraint for a "small" hyperedge. 



Iterative Algorithm for Beck-Fiala Theorem 




(i) Initialization ip(v) <— for all v G V; 




(ii) While V{G) + do 




(a) Find an optimal extreme point solution x to LPdisc- 




(b) If there is a variable with x v = 1 or x v = —1, then set tp(v 


) = x v and set 


B e = B e — x v for each hyperedge e containing v and remove v from G. 


(c) If there exists a hyperedge e G E with e < d, then remove 


the constraint 


for e and remove e from G. 




(iii) Return if). 





Fig. 13.1. Beck-Fiala Discrepancy Theorem 
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13.1.4 Correctness and Performance Guarantee 



We first show that the discrepancy of ip is at most 2d — 1 assuming that the algorithm 
terminates. Consider any hyperedge e. Note that the updates of B e over the course of 
the algorithm ensure that the updated LPs solved in the subsequent iterations continue to 
remain feasible. We prove by induction that B e + disc^(e) = while the constraint for e 
is present, where B e is the current target discrepancy in the linear program and disc^p(e) 
is the current discrepancy in the partial solution constructed so far. This condition holds 
initially as B e = and disc^(e) = since x v = for every vertex v. Whenever we 
set tp(v) = 1 or tp(v) = —1, disc^(e) increases by ip(v) and B e decreases by ip(v) for 
each hyperedge e that contains v, and hence the equality continues to hold. When the 
constraint for a hyperedge e is removed, there are at most d vertices left in e. Since each 
variable v has value — 1 < x v < 1, the equality Ylvee x v = B e = —disc^(e) implies that 
\disc^(e)\ = | ^2 vee x v \ < d. Hence, after the constraint is removed, even if we color the 
remaining vertices arbitrarily, the maximum discrepancy of this edge is strictly less than 
2d. 

To complete the proof of Theorem 13.1.1 we show that the algorithm will terminate 
by a simple counting argument. 

Lemma 13.1.3 Given any extreme point solution x of LP^i sc (G) , at least one of the 
following must hold. 

(i) There exists a vertex v with x v G { — 1, 1}. 

(ii) There exists an edge e £ E with |e| < d. 

Proof Suppose that neither is true. From Lemma 13.1.2 there is a subset of hyperedges 
F C E with \F\ = \ V\. Thus the number of variables |V| is at most \E\. As each hyperedge 
contains strictly more than d vertices, 

]T|e| >d\E\, 

but as each variable occurs in at most d hyperedges we have 

^|e| = ^d £ (t,)<d|y| 

Thus we must have \V\ > \E\ which is a contradiction. □ 

13.2 Rearrangments of Sums 

In this section we will present Steinitz's result on rearrangments of sums in a geometric 
setting using an iterative method. Let V be a subset of vectors in the unit ball B d in the 
d-dimensional Euclidean space. Assume that X^gy v = 0. The question is whether there 
is an ordering v\, V2, ■ ■ ■ , v n of the elements of V such that all partial sums along this order 
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are bounded by a number that only depends on d. Steinitz [156] answers this question 
affirmatively 



Theorem 13.2.1 Given a finite set V C B with X^eV v = ®> ^ ere * s an ordering 
v±,V2, ■ ■ ■ ,v n of the elements of V such that 

k 

1 1 ^2 Vi\ | < d for all 1 < k < n, 

i=i 

where \ \v\\ is the Euclidean norm of v. 



13.2.1 Linear Programming Relaxation 

The approach is not to directly formulate the problem as an optimization problem, instead 
the key idea is to consider an auxiliary linear program. In the following k is a constant 
whose value will be chosen in the range between n and d + 1. 

J2a v v = (13.1) 

J^a v = k-d (13.2) 
< a v < 1 for all v G V (13.3) 

Note that since each vector v G V is d-dimensional, the equality constraint (13.1) is actually 
an abbreviation of d linear equality constraints. 



13.2.2 Characterization of Extreme Point Solutions 

The following lemma is used to "create" a zero-valued variable in the auxiliary linear 
program. 

Lemma 13.2.2 When k = \V\ — 1, given any extreme point solution to the auxiliary linear 
program, there exists a vector v with a v = 0. 

Proof There are \V\ variables in this linear program. By the Rank Lemma there are \V\ 
constraints satisfied as equalities in an extreme point solution. The first two constraints 
contribute d+ 1 equalities. If a v > for all v £ V, then there are at least \ V\ — d—l = k — d 
vectors with a v = 1. Hence, to satisfy the equality constraint (13.2) in the auxiliary linear 
program, the remaining d+ 1 variables must be zero, a contradiction. □ 
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13.2.3 Iterative Algorithm 



The iterative algorithm produces the ordering in reverse order. In each iteration the right 
hand side of equality (13.2) of the linear program decreases, and some vector with coefficient 
zero will be put at the end of the ordering among the current vectors. 

Iterative Algorithm for Steinitz Theorem 

(i) Initialization k = \V\ — 1; 

(ii) While \V\ > d+ 1 do 

(a) Compute an extreme point solution x to the auxiliary linear program. 

(b) Find a vector u with a v = 0. Set Vk+\ ■= u. Remove u from the problem. 
Set k := k - 1. 

(iii) Order the remaining d + 1 vectors arbitrarily to obtain v±, V2, ■ ■ ■ , v<i+i- 

(iv) Return the ordering {v\,V2, ■ ■ ■ , v\v\}- 



Fig. 13.2. Steinitz's Theorem on Rearrangments of Sums 



13.2.4 Correctness and Performance Guarantee 

First we prove that the algorithm will succeed in producing an ordering. There are two 
parts: one is to show that the linear program is always feasible, and the other is to show that 
there is always a vector with coefficient zero. For the first part, initially when k = \V\ — 1 
the solution a v = (\V\ — 1 — d)/\V\ for all v is feasible since YlveV v = ®- N° w suppose 
there is a feasible solution in the current iteration. When k is decreased by one, we can 
obtain a feasible solution by scaling down the current solution by a factor of fc ^^ d . So by 
induction there is always a feasible solution to the linear program. The second part follows 
directly from Lemma 13.2.2. 

Finally we prove that the norm of all partial sums is at most d. This is trivial for 
the first d vectors. Now consider the case when k > d + 1. In the following let a v be the 
coefficient of vector v in the iteration when there are exactly k vectors left; note that we 
can scale down the solution in the last iteration to obtain a feasible solution for k = d+ 1. 

k k k k 

^2v i = ^2v i -^2a Vi v i = ^2(l-a Vi )v i , 

i=l i=l i=l i=l 

where the first equation follows because J2i=i a v t v-i = by the equality constraint (13.1) 
in the linear program. Taking norms and using that 1 — a Vi > and \\vi\ \ < 1 gives 

k k k k 

\\^2,Vi\\ = II 2^(1 - a Vi )vi\\ < 211(1 - a Vi )vi\\ < ~ = k - (k - d) = d, 

i=l i=l i=l i=l 
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where the second last equation follows from the equality constraint (13.2) of the linear 
program. This completes the proof of Theorem 13.2.1. 



13.3 Minimum Cost Circulation 

In this section, we show the integrality of a linear programming relaxation for the minimum 
cost circulation problem. This proof will serve as a basis for the iterative algorithm for the 
minimum cost unsplittable flow problem in the next section. 

In an instance of the minimum cost circulation problem, we are given a directed graph 
D = (V, A) with arc costs c : A — >■ R, capacities u — > R + and a demand function b : V — > R. 
The task is to find a minimum cost flow x : A — > R + such that x(8 out (v )) — x{5 tn {v)) = b v 
for each v G V. Recall that x(F) for F C A is a shorthand for ^2 eeF x e . The vertices with 
b v > are referred to as sources and vertices with b v < are referred to as sinks. Note 
that for feasibility, we must have b v = 0. In the following theorem we prove that when 
the demands and the capacities are multiples of some integer d, then there is an optimal 
flow which is <i-integral. 

Theorem 13.3.1 Consider an instance of the minimum cost circulation problem with d\b v 
(b v is divisible by d) for each v G V and d\u a for each a G A for some d G Z + . Then there 
must exist a minimum cost solution x such that d\x a for each arc a G A. 



13.3.1 Linear Programming Relaxation 

The following is a standard linear programming formulation for the minimum cost circu- 
lation problem, denoted by LP c i rc . 

minimize ^ 

subject to x{5 out {v)) - x{5 in {v)) = b v V v G V 

< x a < u a V a G A 



13.3.2 Characterization of Extreme Point Solutions 

For each vertex v, the corresponding constraint x(5 out (v)) — x(S in (v )) defines a characteris- 
tic vector x(v) in M'" 4 ' with an 1 corresponding to each arc in 5 out (v ) and a —1 corresponding 
to each arc in 5 m (v) and otherwise. The lemma below is a direct consequence of the 
Rank Lemma. 

Lemma 13.3.2 Given any extreme point solution x to LP c i rc with < x a < u a for each 
a G A, there exists a subset W C V such that 
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(i) x{5 out {v)) - x(5 in (v)) = b v for each v G W. 

(ii) The vectors in {x( v ) '■ v 6 ^} are linearly independent. 

(iii) |A| = \W\. 

The following corollary states the property that will be used later. 

Corollary 13.3.3 Given any extreme point solution x to LP c i rc with < x a < u a for each 
a £ A, we must have \A\ < \V\ — 1. 

Proof By Lemma 13.3.2 we have \A\ = \W\. Note that YlvzV x{ v ) = 0, since every arc 
is counted exactly once as a positive term and exactly once as a negative term. As the 
constraints in W are linearly independent, this implies that \W\ < \V\ — 1, and thus the 
corollary follows. □ 



13.3.3 Iterative Algorithm 

We now present the iterative algorithm for the minimum cost circulation problem, which 
will return an optimal solution x with the property that d\x a for each a £ A. 

Iterative Minimum Cost Circulation Algorithm 

(i) Initialization T 0. 

(ii) While V{G) ± do 

(a) Find an optimal extreme point solution x to LP c i rc . 

(b) Fix every arc a with /„ = 0. Add atoT with this value setting and delete 
a from G. Also delete vertices with no arcs incident to them. 

(c) If there exists (v, w) G A such that X( VjW \ = ur VjW \ then fix (y, w) with this 
value setting, add a to T and delete a from G. Update b v b v — X( VjW ) 
and b w <r- b w + X( VjW y 

(d) Find a vertex v with at most one arc a incident on it. Fix a with value x a 
and add a to T . Update G If a is an out-arc (v, w), then update 
b w b w + X( VjW y, otherwise, a = (w, v) and update b w ^— b w — X( VjW y 

(iii) Return P. 



Fig. 13.3. Minimum Cost Circulation Algorithm 



13.3.4 Correctness and Optimality 

Observe that each of the update steps maintains the property that the flow fixed on each 
arc is a multiple of d, since it is either fixed to u a or is fixed to ±b v for some vertex v. 
Optimality of the cost follows from a simple inductive argument. To prove Theorem 13.3.1, 
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it remains to prove that the algorithm terminates by showing that one of the choices in 
the while loop is available in each iteration. 

Lemma 13.3.4 For any extreme point solution x to LP c i rc with < x a < u a , there is a 
vertex v with d(v) = d out (v) + d in {v) < 1. 

Proof Suppose for contradiction that d(y) > 2 for each v G V. Thus we have \A\ = 
\ J2vev d ( v ) ^ 1^1- This contradicts Corollary 13.3.3. □ 

Thus we obtain a d-integral minimum cost flow as an optimal solution proving The- 
orem 13.3.1. 



13.4 Minimum Cost Unsplittable Flow 

In this section we show an application of the iterative relaxation method on the minimum 
cost single source unsplittable flow problem. In an instance of this problem, we are given 
a graph G = (V, A) with edge costs c : A — > R + , edge capacities u : A — > R + , a source 
s £ V and a set of sinks T = {t±, . . . , t^} with demands d±, cfe, • ■ ■ , dfe- The task is to find 
a minimum cost unsplittable flow from the source to the sinks that satisfies the capacity 
constraints. A flow is called unsplittable if the total flow to each sink follows exactly one 
flow path. A flow is called splittable if the total flow to each sink can use multiple paths. 

Theorem 13.4.1 Given an instance of the unsplittable flow problem with di\dj or dj\di for 
each 1 < i,j < k, there exists an unsplittable flow which violates the capacity constraint 
on any arc by at most d max = maxi<j<fc di, with total cost at most the optimal cost of any 
splittable flow that satisfies all the capacity constraints. 

A similar but weaker theorem holds without any assumption on the demands; see the 
exercises. We prove Theorem 13.4.1 by an iterative relaxation algorithm. The underlying 
integral problem is the min-cost circulation problem and we will use Theorem 13.3.1. 



13.4-1 Linear Programming Relaxation 

We obtain the following linear programming formulation, denoted by LP unsp nt, for the 
minimum cost unsplittable flow problem, by casting it as a minimum cost circulation 
problem. Here b v = —d v for each v 6 T and b s = X^eT an d b v = for each v £ 
V\(TUs). 
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minimize 2 c a x a 

a<=A 

subject to x(5 out (v)) - x{5 in {v)) = b v V v e V 

< x a < u a V a € A 

Since the linear program is the same as LP c i rc , we obtain the same characterization as in 
Lemma 13.3.2 as well as in Corollary 13.3.3. 



13.4-2 Iterative Algorithm 

We now present the iterative algorithm proving Theorem 13.4.1. The main idea is to relax 
the capacity constraints on the arcs so that Theorem 13.3.1 can be applied. 



Iterative Minimum Cost Unsplittable Flow Algorithm 

(i) Initialization F «— 0. 

(ii) While V(G) + do 

(a) Let d m i n = min{c£„ : v G T}. Round up u a to the closest multiple of d m i n 
for all arcs a £ A. 

(b) Find an optimal extreme point solution x to LP unsp i it and remove all arcs 
with x a = 0. 

(c) Find a path from from s to v G T where d v = d m i n . Route the flow to v 
on this path. Update T T \ {v} and reduce capacities on the arcs on 
the path by d m in- 

(iii) Return the set of flow paths discovered. 



Fig. 13.4. Minimum Cost Unsplittable Flow Algorithm 



13-4-3 Correctness and Performance Guarantee 

In each step, when a flow is routed to some sink v, Theorem 13.3.1 implies that each arc 
carries flow which is at least d v = d m i n . Thus it is possible to route d m { n flow on this path 
as required by the algorithm. The optimality of the cost follows from a standard inductive 
argument, since at each step we only relax the capacity constraints. We now show that the 
capacity on each arc is violated by at most d max , completing the proof of Theorem 13.4.1. 

Lemma 13.4.2 The iterative algorithm for the minimum cost unsplittable flow problem 
returns an unsplittable flow such that the capacity on each arc is violated by at most d max . 
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Proof The routed flow is unsplittable by construction. The capacities are violated in 
Step (ii) (a) when they are rounded up. Let the demands be d\ < di < . . . < dk in the 
increasing order. Observe that di^l ■ ■ • \dk- We now show by induction that when the 
demands d\, . . . ,di have been routed, the capacity on each arc is at most di more than 
its initial capacity and is a multiple of di for each 1 < i < k. The claim clearly holds for 
i = 1. By the induction hypothesis we assume the claim is true for i — 1, i.e, the capacity 
is violated by at most d{-\ and is a multiple of di-\. While routing di, the capacity is 
increased to a multiple of di. Since di-\\di, the increase in the i th step is bounded by 
di — di-\ which bounds the total violation at the end of i th step by di as required. □ 



13.5 Bin Packing 

In this section we show the Karmarkar-Karp algorithm for the bin packing problem. This is 
one of the earliest and is still one of the most sophisticated instances of using the iterative 
technique for an approximation algorithm. In an instance / of the one-dimensional bin 
packing problem, we are given n items each of which has a size between and 1. The 
objective is to pack the items into a minimum number of unit-size bins. Let opt(I) denote 
the number of bins required in an optimal solution to instance /. We present the following 
theorem due to Karmarkar and Karp. 

Theorem 13.5.1 There is a polynomial time algorithm which returns a solution with at 
most opt (I) + 0(log 2 opt(7)) bins. 



13.5.1 Linear Programming Relaxation 

A natural linear programming relaxation is to require that each item is put in at least 
one bin and each bin can pack items of total size at most one, but this relaxation has a 
(multiplicative) integrality gap of 2; see exercises. To obtain an additive approximation, 
we need a so-called configuration linear programming relaxation for the problem. Consider 
an instance / of the bin packing problem with n(I) items of m(I) different types. For 
1 < i < m(I), let hi be the total number of items of type i and Sj be the common size of 
these items. Let T±, . . . ,Tjy be all the possible configurations in which a single bin can be 
packed: 

m 

{Ti, ...,T N }:= {(h, k m{1) ) £Z™:^ hs t < 1}. 

i=l 

Note that N can be exponential in n(I) and m(I). Let Tj = (tji, . . . , tj m m) where 
tji denotes the number of items of type i in configuration j. The configuration linear 
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programming relaxation for the bin packing problem is as follows. 

N 

minimize Xj 
3=1 

N 

subject to ~^2^ji x j — h Vl<i<m(/) 

3=1 

xj > V 1 < j < N, 

where the constraints ensure that the configurations chosen contain at least hi items for 
each type i. This linear programming has exponentially many variables, but there is a 
polynomial time algorithm to compute a fractional solution differing from the optimum by 
at most 5. Consider the dual of the linear program. 

m(I) 

hiVi 

i=l 

m(I) 

Y^tjiyi < 1 Vl<j<iV 
i=i 

Vi > Vl<i<m(J), 

The dual program has m variables but exponentially many constraints, but if there is a 
polynomial time separation oracle to determine if a given y is a feasible solution, then the 
dual program can be solved by the ellipsoid algorithm. The separation problem for the dual 
linear program is to determine, given y, if there exists a configuration T = {ii, £2, ■ ■ ■ , t m } 
so that Y^I=i UVi > 1- This is equivalent to the following maximization problem on the 
variables U. Recall that denotes the size of items of type i. 



1 

Z + Vl<i<m(7), 

where the constraint is the definition of a configuration. This is a knapsack problem where 
Si and yi correspond to the size and profit of item i respectively. So the separation problem 
is equivalent to a knapsack problem, which is NP-hard to solve optimally. Nevertheless, 
if there is a polynomial time weak separation oracle to determine whether a given y is a 
feasible dual solution with error at most 5 on the dual objective function or y is an infeasible 
dual solution, then the dual program can be approximated by the ellipsoid algorithm with 
an error at most 5. 

In the following we sketch how to obtain an approximate solution to the (primal) 
configuration linear program using the ellipsoid algorithm. For the dual program, we 



maximize 



subject to 



m(J) 

maximize y{ti 
i=i 
m(I) 

subject to s iti ^ 

i=l 

U e 
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can solve the weak separation problem by rounding each yi down to the nearest rational 
number that is a multiple of and then use a dynamic programming algorithm to solve 
the resulting knapsack problem optimally in polynomial time. Using the ellipsoid algorithm 
with the weak separation oracle, we can then obtain a solution y* to the dual problem with 
y*b > OPTdual — S, where b is the vector of the number of items of each type and OPTdual is 
the optimal value of the dual program. Let T[,T^ . . . ,T' N , be the bin configurations that 
appeared as a separating hyperplane during the execution of the ellipsoid algorithm, where 
N' is bounded by a polynomial in n(I) and m(I). Consider the dual program restricted to 
the constraints that correspond to T[, . . . , T' N , and let the optimal value of this restricted 
dual program be OPT dual . Then y*b > OPT dual — 5, since the weak separation oracle can 
always give the same answer as for the original problem. To obtain an approximate solution 
to the primal program, we obtain an optimal solution to the restricted primal program in 
which we use only the variables that correspond to the configurations in , . . . , and 
delete all other variables; in other words, we obtain an optimal solution to the dual of the 
restricted dual program. Since there are only polynomially many variables and constraints, 
this resticted primal program can be solved optimally in polynomial time. Let OPT' primal 
be the optimal value of the restricted primal program. Then we have 

O pT 'primal ~S = OPT dual - 5 < y*b < OPT dual = OPT prima/ . 

Therefore an optimal solution to the restricted primal program is an approximate solution 
to the primal program with additive error at most S. 

13.5.2 Characterization of Extreme Point Solutions 

The following lemma is a direct consequence of the Rank Lemma. 

Lemma 13.5.2 Given any extreme point solution to the configuration linear program, there 
are at most m(I) nonzero variables, where m(I) is the number of different types of items 
in instance I. 



To illustrate the use of Lemma 13.5.2, we show a very simple algorithm with a 
good performance guarantee when the number of different types of items is small. Let 
lin(J) denote the optimal value of the configuration LP associated with instance /. Let 
SIZE(/) = Yli^P s ibi be the sum of the sizes of all the items in instance I. 

Lemma 13.5.3 opt(J) < lin(J) + m(f 2 )+1 . 

Proof Let x be an optimal extreme point solution of the configuration LP for instance I. 
Then, by Lemma 13.5.2, x has at most m(I) nonzero variables. We open [xj\ bins with 
configuration j for each j. The remaining items form an instance I'. Let fj = Xj — [xj\. 
Then size(J') < lin(I') = ^ ■ fj. We now find a packing for instance I' of cost at most 

SIZe(J') + m ^ +1 . A packing for instance I' of cost at most m(I) can be constructed by 
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using one new bin for each configuration j with nonzero fj, and then removing excess items 
(the items that appear in more than one configuration). On the other hand, any greedy 
packing algorithm will give a solution for instance I' of cost at most 2size(/') + 1, since each 
bin, except possibly one, will be at least half full. Hence, the better of these two packings 
has cost at most the average, which is SIZE(I') + m ^ +1 < £\ j- _|_ m ( I )+ 1 ^ w hj cn j n turn 

gives a packing for instance / with cost at most ^ ■ [xj\ + fj + m ^2 +1 = lin(J)+ m( - I ^ +1 ; 
proving the lemma. □ 



13.5.3 Defining Residual Problems: Grouping and Elimination 

Given Lemma 13.5.3, one would like to reduce the number of distinct item sizes of the 
input instance. The idea of grouping is to divide the items into groups of similar sizes, and 
create a residual problem by increasing the sizes of each item to the largest item size in 
its group, so as to decrease the number of distinct item sizes in the residual problem. By 
doing so, however, the total item size of the residual problem and hence the optimum of 
the residual problem increases, and so there is a tradeoff in choosing the parameters. The 
main idea of the Karmarkar-Karp algorithm is to define the residual problem iteratively, 
so that in each iteration the number of distinct item sizes decrease by a constant factor, 
while the optimum in the residual problem increases by only an extra 0(log(oPT")) number 
of bins. Applying this procedure inductively will lead to Theorem 13.5.1. 

The following is a simple method to bound the optimum of the residual problem. Let 
/ and J be two bin packing instances. We write / -< J if there is an injective function / 
mapping items in I into items in J so that SIZe(o) < slZE(/(a)) for each item a £ I, where 
SlZE(a) is the size of item a. Clearly, if I ^ J, then opt(7) < OPT(J), lin(I) < LIN (J) 
and SIZE(I) < size(J). This method will be used throughout to analyze the performance 
of the grouping techniques. 

13.5.3.1 Linear Grouping 

The following process is called linear grouping with parameter k. Let / be an instance 
of the bin packing problem and let k be a positive integer. Divide the set / into groups 
Gi, G2, ■ ■ ■ , G q so that G\ contains the k largest items, G2 contains the next k largest items 
and so on. Hence G\ >z G2 <z • • • >r G q and \Gi\ = k for all groups except the last group. 
Let G\ be the multi-set of items obtained by increasing the size of each item in group Gi 
to the maximum size of an item in that group. Then G\ >z G' 2 >z G2 >r • • • h G' q >z G q . 
Let J := Uf =2 G- and J' := Gy. Then J < I ^ J U J'. Note that the items in J' can be 
trivially packed in k new bins, by using one new bin for each item. Therefore, we have the 
following after linear grouping. 

Lemma 13.5.4 After linear grouping with parameter k, we have 
• opt(J) < opt (J) < opt(J) + k, 
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• lin(J) < lin(7) < lin(J) + k, 

• size(J) < size(I) < size(J) + k. 

13.5.3.2 Geometric Grouping 

The following refinement of linear grouping is called geometric grouping with parameter 
k. This is the key idea in the Karmarkar-Karp algorithm to reduce the number of distinct 
item sizes iteratively. 

Let I be an instance of the bin packing problem. Let a(I) denote the size of the 
smallest item in instance I. For < r < |log 2 H7n J > ^ Ir be the instance consisting of 
those items from I whose sizes are in the interval [2~^ r+l \ 2~ r ). Let J r and J' r be the 
instances obtained by applying linear grouping with parameter k • 2 r to I r . Let J := U r J r 
and J' : = U r J' r . 

Lemma 13.5.5 After geometric grouping with parameter k, we have 

• opt(J) < opt(7) < opt(J) + k\\og 2 

• lin(J) < lin(J) < lin(J) + A;[log 2 

• size(J) < size(I) < size(J) + A;[log 2 ^jy], 

• m(J) < |size(7) + riog 2 ^7)l- 

Proof From Lemma 13.5.4 it follows that 3 r <l r < J T ^J' r and thus J <1 < JU J' '. Hence 
opt(J) < opt(J) < opt(JuJ') < opt(J) + opt(J'). Note that opt(J') < Y, r OPT ( J 'r)- 
Each J' r contains at most k ■ 2 r items each of size less than 2~ r . Hence J' r can be packed 
into at most k bins. Thus OPT(J') < fc[log 2 ^jy], and therefore OPT(J) < opt(J) < 
OPT(J) + A;[log 2 ^jyl • The proofs of the next two inequalities are similar. 

For each r, since we apply linear grouping with parameter k • 2 r , all except the last 
group in J r have k ■ 2 r items and the items in each group are of the same size. Also, each 
item in I r has size at least 2~( r+1 \ and thus we have 

siZE(/ r ) > 2-( r+1 > • n{I r ) > 2-( r+1 ) ({m{J r ) - 1) • k ■ 2 r ) . 

Therefore m(J r ) - 1 < |siZE(/ r ) and thus 

2 1 

m(J) < -size(/) + f!og 2 — ]. 

□ 

13.5.3.3 Elimination of Small Items 

In the geometric grouping process, the performance depends on the smallest item size. 
In the following we show that we can eliminate items of small sizes without affecting the 
approximation performance ratio much. Let I be an instance of the bin packing problem. 
Let g be a real number between and 1. We say an item is large if its size is larger than | 
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and small otherwise. Consider a process which starts with a given packing of large items 
into bins, and then inserts the small items, using a new bin only when necessary. If the 
cost of the given packing of the large pieces is C, then the cost of the packing resulting 
from the process is at most max{C, (1 + g)oPT(I) + 1}; see the exercises. 



13.5.4 Iterative Algorithm 

With the grouping technique developed, we present the iterative algorithm in Figure 13.5. 
The parameters k and g will be set as k = 4 and g = qtwttt- 



Iterative Bin Packing Algorithm 

(i) Eliminate all small items of size at most g. 

(ii) While size(J) > 1 + j\ [log 2 ±] do 

1 k 9 

(a) Perform geometric grouping with parameter k to create instance J and J'. 
Pack J' using at most fc|~log 2 |] new bins. 

(b) Compute an optimal extreme point solution x to the configuration LP on 
instance J with error at most 1. 

(c) For each j with Xj > 1, create [xj\ bins with configuration j and remove 
the items packed from the problem. 

(iii) Pack the remaining items using at most 2 + — ^ [log 2 ^] bins. 

(iv) Insert the small items eliminated in Step 1, using new bins only when necessary. 

Fig. 13.5. Karmarkar-Karp Bin Packing Algorithm 



13.5.5 Correctness and Performance Guarantee 

To prove the correctness and the performance guarantee, we will bound the number of 
iterations of the algorithm (in particular it will terminate) and the number of bins used 
by the algorithm. Let t be the number of iterations of the algorithm. For 1 < i < t, let Ii 
be the instance at the beginning of iteration i, and Jj, J[ be the instances resulting from 
geometric grouping on /j, and let Xi and Y{ be the number of bins created in Step (ii) (c) 
and Step (ii) (a) in iteration i of the algorithm. 

Lemma 13.5.6 The iterative algorithm will terminate in at most O(lnn) iterations. 

Proof Note that the total size of the residual problem decreases geometrically: 

2 1 

SIZE(/ i+ i) < LIN(/ i+ i) < J2(xj - [Xj\) < m(Ji) < -SIZE(ii) + flog 2 -1, 



208 



where the second last inequality follows from Lemma 13.5.2 and the last inequality follows 
from Lemma 13.5.5. Therefore 

SIZE(/ m ) < (^SIZE(J) + [log 2 h [1 + \ + ■ ■ ■ + (^r 1 ] < (^SIZE(J) + * [log 2 h . 

y ■• k y 

Since SlZE(I t ) > 1 + 7^2-[log 2 \~\, this implies that (|)*SIZe(I) > 1 and hence 

lnsiZE(J) 
t< : fc V ^ +1 = lnw . 
In | 

Therefore the running time of the algorithm is polynomial. □ 

Lemma 13.5.7 The total number of bins used by the algorithm is at most OPT (I) + 
0(ln 2 OPT(I)). 

Proof To bound the number of bins used by the algorithm, we note that LlN(ij+i) < 
LlN(Jj) + 1 — Xi < LlN(ij) + 1 — Xi by Step (ii) (c) of the algorithm, which implies that 

t 

Yj X i< LIN(/)+t. 

i=i 

By Lemma 13.5.5 we have for each 1 < i < t 

Yi < fclloga^l. 

Note that Step (iii) is always possible by any greedy algorithm. The number of bins 
produced by the algorithm after Step (iii) is at most 

V^ + t-y i + 2 + -^ T [i og2 -l. 

After inserting the small items, the total number of bins used by the algorithm is at most 
the maximum of (1 + g)o¥ r r{I) + 1 and 

/T , /lnslZE(J) \/ , ri 1,\ „ 2 ri l n 

OPT(/) + ( ln fe + l ) \} + k ^ „l) + 2 + rnN 2 -1, 



because £* =1 X i ^ LIN ( J ) ^ OPT(J) and t < ( lnSIZ f (J) + 1) and ^ < fc[log 2 ^l. By 

In 2 9 

setting k = 4 and g = SIZ e(j) — n' anc ^ n °t m g that opt(T) > size(J), we see that the 
total number of bins is at most opt(7) + 0(ln 2 opt (I)), proving theorem 13.5.1. □ 



13.6 Iterative Randomized Rounding: Steiner Trees 

In this section, we present a recent application of the iterative rounding method augmented 
with randomized rounding for the classical undirected Steiner tree problem. The Steiner 



209 



tree problem is defined on an undirected graph G = (V, E) with costs c on the edges and 
the vertex set partitioned into terminals R and non-terminals or Steiner nodes V \ R. A 
Steiner tree is an undirected spanning tree that must span the terminals and can span 
any subset of the Steiner nodes. The Steiner tree problem is to find one of minimum cost, 
and is known to be APX-hard. The methods from Chapter 10 give a 2-approximation 
algorithm for this problem (using the original method of Jain that introduced the iterative 
method) as well as its various generalizations. 

The basic Steiner tree problem however had a simple 2-approximation algorithm 
before the iterative rounding method of Jain. First, observe that we may work with the 
metric completion of the costs on the edges in solving the problem: this completion replaces 
the graph with a complete graph where the cost of any new edge is the cost of the shortest 
path under the original costs between the endpoints of the edge. It is not hard to see that 
edges of any optimal solution will be retained in the completion, and that any Steiner tree 
in the completion can be converted to one in the original graph by replacing any new edges 
chosen by the corresponding shortest path. Thus we can assume that the input to the 
Steiner tree instance is a metric without loss of generality. 

The classical 2-approximation for the Steiner tree problem starts with the metric 
completion restricted only to the terminal nodes in R, and computes a minimum spanning 
tree on it. To get a feasible solution, we replace the edges in the spanning tree by the 
corresponding paths as above. To see that this solution is indeed a 2-approximation, it 
suffices to demonstrate a solution in the metric completion on R of cost at most twice 
the optimal. To do this, consider an optimal Steiner tree, double every edge, and take 
an Eulerian tour around the "perimeter" of a planar representation of this doubled tree. 
This walk can start at any terminal and visit every other terminal once and return to the 
starting terminal: short-cutting the segments between terminals will give a cycle in the 
metric completion on R of cost no more than twice that of the optimal Steiner tree. Thus 
we have demonstrated the existence of a solution of cost at most twice the optimal which 
is a cycle on R made of \R\ segments. Since we only require a tree, we can discard the 
costliest segment and thus argue that there exists a path of cost at most 2(1 — ^) times 
the optimal, completing the proof. 

We now present an improvement to the above algorithm based on building up the 
optimal solution not using single edges denoting paths that join pairs of terminals but 
instead using subtrees that connect more than two terminals optimally. Such subtrees are 
called full components, and we start with the above minimum spanning tree solution and 
identify subsets of more than two vertices that can be connected by optimal full components 
on them. We then include these components and discard a few redundant edges in the 
spanning tree and proceed until we have substantially reduced the cost of the spanning tree. 
The initial application of this idea was due to Zelikovsky [165] and subsequently improved 
in [17, 144]. We describe a recent improvement in this line of attack which chooses the full 
components iteratively and randomly based on an LP relaxation. 
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13.6.1 Linear Programming Relaxation 

A full component on a subset K of the terminals is a tree whose leaf set is K and internal 
nodes are Steiner vertices. Note that the edges of any Steiner tree can be partitioned in 
a unique way into full components. The goal of the LP relaxation below is to describe 
the Steiner tree problem as the choice of its optimal full components. Note that a set of 
full components on sets K\, K2, . . . , K r forms a Steiner tree exactly when the hypergraph 
formed by these subsets as hyperedges on R is a hyper-spanning tree: namely, no pair of 
them has an intersection of more than one terminal, and there is a path between any pair 
of terminals following hyperedges. This leads to the following hypergraphic LP relaxation: 
let F(K) denote the minimum cost full component for terminal set K C R and let Ck 
be its cost. The hypergraphic relaxation for minimum-cost Steiner tree below is similar to 
the subtour LP for spanning trees in its design. 

minimize Ck %k 

K 

subject to x K (\KnS\-l) < \S\-1 

K:KnS^9 

Y,x K (\K\-l) = \R\-1 

K 

X K > 

An integral feasible solution of the above LP gives the full components of a Steiner tree. 
Our goal is to bound the integrality gap of the above relaxation. 

Theorem 13.6.1 The integrality gap of the hypergraphic relaxation for undirected Steiner 
tree is at most 1 + 

To convert the proof of the theorem into a rounding algorithm, one needs to use the 
methodology of arguing that by restricting oneself to sufficiently large full components in 
the above description, one can get sufficiently close to the LP value: formally, to get a 
(1 + e)-approximation to the above LP relaxation, it suffices to use full components of size 
bounded by a (large) constant for every fixed constant e > 0. The details of this claim 
form the basis of the r-restricted full component method which are non trivial, and can be 
found in [80]. 

Finally, we need to show that the above formulation even with bounded-size full 
components is polynomially solvable. For this, we can devise a separation oracle for a 
given fractional solution. We build an auxiliary multigraph on the terminal set R as 
follows: for every full component set K with positive value of xk in the support that is 
supplied to the separation oracle, we add the edges of an arbitrary spanning tree on the 
nodes of K to this auxiliary graph and give each of these edges a fractional value of xk- 
We now claim that the resulting fractional multigraph solution is feasible for the subtour 
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relaxation for a spanning tree on the nodes of R if and only if the supplied solution x is 
feasible for the hypergraphic relaxation. The proof of this claim is very similar to that of 
the proof of the Bridge Lemma 13.6.2 below and is deferred to the Exercises. 



13.6.2 Iterative Randomized Rounding 



To describe the algorithm, we need some notation first. Define the loss of full component 
F(K) on K C R, denoted by LOSS(K) to be the minimum-cost subset of F(K)'s edges 
that connect the Steiner nodes in it to the terminals. We use loss(K) to denote the cost of 
the edges in LOSS(K). The loss- contracted full component of K, denoted by TLC(K), is 
obtained from F(K) by contracting the edges in LOSS(K), or using the contract notation, 
we have TLC(K) = F{K) / LOSS(K). Note that TLC(K) is a spanning tree on the 
terminal set K and its edges do not correspond to edges in the original graph. For clarity 
we denote the edge set E{F(K)) \ LOSS(K) by LC(K). Since the edges in TLC(K) are 
precisely the edges remaining in LC(K), there is a natural bijection between these two 
edge sets. We emphasize that we use two different notations for them only to distinguish 
that TLC(K) is an auxiliary spanning tree on the set K while LC{K) is a subset of edges 
with endpoints in the full component of K including some Steiner nodes. 



Fig. 13.6. In figure (a), the graph F(K) is a full component with terminals K = {a,b,c,d} and 
Steiner vertices {x,y}. The bold edges {a, x} and {y,c} form LOSS(K). In figure (b), the graph 
TLC(K) is shown obtained after contracting LOSS(K) 

Since contraction is involved in several stages of the algorithm below, we may have 
different auxiliary edges between a pair of terminals (due to loss contraction of different 
full components) but we retain all such edges as parallel edges in the analysis. 

The idea of the iterative randomized algorithm below is to start with a MST on 
the terminals and iteratively choose a full component on K with probability proportional 
to the value of its choice variable, and add all of its LOSS(K) edges and some of the 
remaining loss-contracted edges LC(K) to the final solution. An auxiliary spanning tree 




b A a , x ) Aw} , d 
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(b) 
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on the whole terminal set R is used to track which portion of the loss-contracted edges 
from each iteration are retained in the final solution. The candidates for inclusion in this 
auxiliary tree are precisely the auxiliary edges induced in TLC(K). Furthermore, as we 
start retaining some of these intermediate loss-contracted edges, we can drop some original 
spanning tree edges which reduce the cost of the MST that is retained. This auxiliary 
spanning tree that records the drop in the value of the initial solution is denoted Tj at 
iteration i in the algorithm below. This charging device allows us to formulate a clean 
recurrence for the reducing value of Tj in expectation, and hence bound the total cost of 
the solution in the end. The algorithm is shown in Figure 13.7. 



Iterative Randomized Loss Contraction Algorithm 




(i) 


Initialization T x <- MST(R). 




(ii) 


Let x be an optimal solution to the hypergraphic LP relaxation. 


Let M be the 




smallest value that is at least Y1k x k such that t = M In 3 is an integer. 


(iii) 


For 1 < i < t do 






(a) Sample Ki with probability for each full component K 


(and choose no 




tree with probability (1 — J2k x k/M)). 






(b) T l+1 <- MST{Ti U TLC(Ki)). 




(iv) 


Return any Steiner tree in ALG = T t+1 U [f i=1 LOSS(Ki). 





Fig. 13.7. Iterative Randomized Rounding Algorithm for Steiner Tree 



In the last step, when we refer to edges in T t+ i, notice that this may include some 
auxiliary edges from TLCiKi) for some iteration in Step (iii)(b). However, due to the 
bijection between these edges and the corresponding real edges in LC(Ki), we actually 
include the real edges in LC(Ki) in this step. 

We argue that T t+ \ U(J* =1 LOSS(Ki) contains a Steiner tree on R. In particular we 
argue that for every pair of terminals joined by an edge in T t+ \, there is a path between 
them in the union of the real edges corresponding to the auxiliary edges in this tree plus the 
edges in (Ji=i LOSS(Ki). Note that if an edge in T s+ i is an original edge from T\, there 
is nothing to prove. Else, it is a new auxiliary edge introduced as part of some TLC(Ki). 
However the corresponding real edge in LC(Ki) along with the edges in LOSS(Ki) contain 
a path between these two endpoints by construction, proving our claim. 



13.6.3 Correctness and Performance Guarantee 

The main idea of the analysis is to bound the cost of the set of edges ALG in expectation 
over the random component choices of the algorithm, by its constituent parts of the final 
spanning tree T t+ \ plus the loss values of the components contracted on the way. We do 
this in two steps. The cost of the spanning tree is tracked using a recursion relating the 
cost of the tree Tj+i to that of the tree in the previous iteration Tj, and then telescoping 
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the result using a key lemma termed the Bridge Lemma. The expected loss is bounded by 
a simple argument relating it to the cost of the full components from which it is contracted. 
Together, these two bounds give the final bound on the expected cost of the edges in ALG. 

13.6.3.1 Preparations 

We start by defining the drop of a full component K with respect to a terminal spanning 
tree T (i.e., a tree spanning exactly the terminals). Recall from the definition of node con- 
traction that T/K denotes the (multi)graph obtained by identifying the terminals spanned 
by K into a single node. Define 

DROPt(K) = E(T) \ E(MST(T/K)) 

and let dropT(K) be the cost of the edges in DROPt(K). Note that these are precisely the 
edges in T that can now be dropped given an alternate means of connecting the terminals 
in K via a full component on them, hence the name. See Figure 13.8 for an illustration. 




{a, b, c, d} 

(a) (b) (c) 

Fig. 13.8. In figure (a), we show a terminal spanning tree T on the terminals {a,b,c,d,e, /} 
with edges {ea, eb, ef, fc, fd} along with a full component on K = {a, b, c, d} with edges 
{xa,xb,xy,yc,yd}. In figure (b), the dashed edges form DROPt(K). In figure (c), we show 
the graph TDROP T (K) U E(T) where the TDROP T {K) edges are dashed. Observe that each 
edge / € TDROPt(K) is the heaviest edge in the unique cycle in T U {/}. 

The Bridge Lemma lower bounds the expected drop. 

Lemma 13.6.2 Given a terminal spanning tree T and a feasible solution x to the hyper- 
graphic LP formulation, 

^2x K drop T (K) > c(T). 
K 

Proof The proof of this lemma uses the close relation between the hypergraphic relaxation 
and the subtour elimination LP we encountered in Chapter 4 for spanning trees, presented 
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again below. 



minimize 




subject to 



z(E(S)) < \S\-1 
z{E(V)) = \V\-l 

Ze > 



V / 5 c V 



Veefi 



The proof idea is to convert the solution x for the hypergraphic relaxation into a 
solution z for the subtour LP on the vertex set of terminals R. For this we introduce 
some auxiliary edges on the vertex set R. In particular, for each full component K such 
that xk > 0, contracting the edges in E(T) \ DROPt(K) and retaining the labels of the 
terminals in K in the contracted graph, we get a spanning tree TDROPt(K) on the node 
set K. Moreover, there is a bijection between the edges of this spanning tree and the edges 
in DROPt(K) (just as was the case between TLC(K) and LC(K)). Give each of these 
edges in TDROPt(K) (now running between nodes of K) a z- value of xk , making parallel 
edges between pairs of terminals as needed. Let this set of auxiliary multiedges that are 
given z-values be F. We see that 



Note that we introduced exactly \K\ — 1 edges for each full component K, and for any 
S C R, at most \SP\ K\ — 1 of these edges in F have both ends in S since these edges form 
a tree on K. Therefore, x being feasible for the hypergraphic relaxation implies that z is 
feasible for the subtour relaxation. Thus the left-hand quantity in the lemma has cost that 
of a feasible solution to the subtour relaxation on the graph of the auxiliary edges. Note 
that the auxiliary edges corresponding to full components of size two generate the original 
edges of T in this auxiliary graph as well. 

By Theorem 4.1.1, since the subtour relaxation is the convex hull of integral spanning 
trees in the graph made of the edges in E(T) U F, the cost of any tree is at least that of 
the minimum-cost spanning tree in this auxiliary graph. But consider any edge / € F, 
which corresponds to some auxiliary edge in TDROPt(K) for some full component K, 
and hence corresponds to an edge in DROP(K). By the definition of DROP(K), the 
edge / was dropped from the original tree T in choosing MST(T / K), and hence was the 
costliest edge in some cycle in T/K. Thus, when we add this auxiliary edge / between its 
endpoints in TDROPt(K) in the tree T, it still continues to be a maximum cost edge in 
the unique cycle formed in T U /. Since this is true for every / £ F, the minimum cost 
tree in E(T)L)F is T itself. See figure 13.8(c) for an illustration. This completes the proof. 



We need two more observations before we commence the proof. 

Lemma 13.6.3 The value of T\ = MST(R) the initial terminal spanning tree, is at most 
twice the optimal value of the hypergraphic LP relaxation for Steiner tree. 




K 



□ 
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The proof of this lemma follows the short-cutting argument in the beginning of this sec- 
tion, but on the full components in the optimal fractional solution for the hypergraphic 
relaxation, and showing that this doubled short-cut solution is feasible for the subtour LP 
relaxation for spanning trees in the metric completion of R. The details are left as an 
exercise. 

Lemma 13.6.4 For any full component K, we have loss(K) < Ck/2- 

Proof First we assume without loss of generality that every internal Steiner node has degree 
three in K (else we can make it so by splitting higher degree nodes into degree-three nodes 
and connecting them in a tree-like fashion with zero-cost edges). Now root the tree at an 
arbitrary leaf and let every internal node choose the cheapest of its two children. These 
chosen edges give paths from every Steiner node to a terminal leaf, and have cost at most 
twice that of the full component, proving the lemma. □ 

13.6.3.2 Inductive Analysis 
We now give the proof of Theorem 13.6.1. 

Proof We bound the cost of the terminal spanning tree Tj+i based on that of Tj. One 
way to derive this spanning tree is to add in edges of LC(Ki) and drop the edges in 
DROP Tx {Ki). Thus we have 

c{T i+l ) < c{Ti) - dropr^Ki) + c(LC(K i )). 

By linearity of expectation, we can derive the expected value based on the distribution 
from which Ki is drawn as follows. 

E[c(T i+1 )} < E[c(Ti)] - ^^Kdrop^K) + ^ £ x K c(LC(K)). 

K K 

Let Ip* = Y^k x kCr and loss* = Y1 K xxloss(K), and since LC(K) = Ck — loss(K), we 
have 

E[c(T i+1 )} < E[c(Ti)] - ^J2 x K dr °PT,(K) + (lp* ~ loss*)/M. 

K 

Applying the bridge lemma 13.6.2 to bound the second term, we get the following: 

E[c(T t+1 )} < (1 - ^)E[c(T$ + (lp* - loss*)/M. 
By induction, we can unravel this recurrence as follows: 

E[c(T t+1 )] < (1 - i)'^)] + (lp* - loss*)(l - (1 - JL)«). 
Now we use the bound from Lemma 13.6.3 to get the following bound. 

E[c(T l+1 )] < lp*(l + (1 - JL)*) _ loss*(l - (1 - JL)«). 
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This takes care of the first component of the edges in ALG. For the second component, 
namely the edges in \j\ =l LOSS(Ki), we have the following bound: 

t t t 

E[loss(Ki)} = ^ J2x K loss(K) /M = ^2 loss*/M = t ■ loss*/M. 
i=l i=l K i=l 

Adding the two components, we get 

E[c(ALG)] = E[c(T t+1 )]+t-loss*/M 

< va + a-i)')+i-*(s + a-i)'-i) 

< Wd + ^). 

Here the second inequality uses Lemma 13.6.4, and the last inequality follows from the 
fact that \ + \e~ x + § is minimized at x = In 3 taking on value 1 + This is also why 
we chose t = Mm 3. □ 



Notes 

The discrepancy theorem is due to Beck and Fiala [14]. Our presentation of the Steinitz's 
result follows that of Barany [12], who attributes this proof to Grinberg and Sevastyanov [79]. 
Steinitz's original proof [156] also used linear dependencies, and gave constant 2d instead 
of d. Steinitz's result has many applications in mathematics (see e.g. [12]), as well as in 
designing approximation algorithms for scheduling problems (see e.g. [13] and the refer- 
ences in [12]). Some other problems about sums of ±1 signs can also be solved by a similar 
iterative method, see [12]. 

The single source unsplittable flow problem was first introduced by Kleinberg [99], 
who gave a constant factor approximation algorithm for the unweighted problem. This was 
improved by Dinitz, Garg and Goemans [42], who gave an algorithm for the unweighted 
problem which finds an unsplitting flow that violates the arc capacities by at most d max . 
Kolliopoulos and Stein [100] used the iterative relaxation idea to obtain a bicriteria ap- 
proximation algorithm for the weighted problem, which returns a solution with cost at 
most twice the optimum and the arc capacities at most thrice that of the splittable flow. 
This is improved by Skutella [152] to get an approximation algorithm with optimal cost 
and the same guarantee on the arc capacities. 

The bin packing result is from the work of Karmarkar and Karp [93], which builds on 
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the asymptotic PTAS by Fernandez de la Vega and Lueker [53] and develops the iterative 
relaxation technique for the configuration linear program of the problem. 

Our description of the Steiner tree approximation algorithm is based on the work 
of Bryka, Grandoni, Rothvofi and Sanita [23]; we closely followed the treatment due to 
Chakrabarty, Konemann and Pritchard [28]. 

Exercises 

13.1 (Bednarchak, Helm [15]) The bound on the Beck-Fiala theorem can be improved 
to 2d — 3. Can you prove this improved bound by refining the iterative method? 

13.2 (Skutella [152]) Given any instance of the unsplittable flow problem, let / be a min- 
imum cost splittable flow. Give an efficient algorithm that returns an unsplittable 
flow with cost at most that of / such that the total flow on each arc a is at most 
2f a + d max . (Hint: Reduce the general problem to the problem in Theorem 13.4.1.) 

13.3 Show that the natural LP relaxation for the bin packing problem has a multiplica- 
tive integrality gap approaching two. On the other hand, show how a greedy bin 
packing algorithm achieves a packing of instance / into at most 2size(7) + 1 bins. 

13.4 Show that the greedy procedure to pack small items defined in Section 13.5.3.3 uses 
at most max{C, (1 + g)oPT(I) + 1} bins. 

13.5 Show that the bin packing problem is a special case of the single source unsplittable 
flow problem. 

13.6 Show that the generalized assignment problem is a special case of the single source 
min-cost unsplittable flow problem. 

13.7 Prove Lemma 13.6.3 by showing that an optimal solution to the hypergraphic LP 
relaxation for Steiner trees can be doubled and converted to a feasible solution to 
the subtour relaxation for the minimum spanning tree problem on the terminals of 
the Steiner tree problem. 

13.8 Prove that the hypergraphic LP relaxation for Steiner trees with full components 
of up to a fixed constant size can be solved in polynomial time. Use the method 
outlined in the Chapter of converting this to testing separation of a corresponding 
point in the subtour elimination formulation for spanning trees on an auxiliary 
multigraph on the terminal set, as in Lemma 13.6.2. 
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Summary 



t 

We have described an iterative method that is versatile in its applicability to show- 
ing several results in exact optimization and approximation algorithms. The key step in 
applying this method uses the elementary Rank Lemma to show sparsity of the support 
of extreme point solutions for a wide variety of problems. The method follows a natural 
sequence of formulating a tractable LP relaxation of the problem, examining the structure 
of tight constraints to demonstrate an upper bound on the rank of the tight subsystem 
defining the extreme point, using the Rank Lemma to imply an upper bound on the sup- 
port and finally using this sparsity of the support to find an element in the support with 
high (possible fractional) value. 

The two key steps in the method are upper bounding the rank of the tight constraints, 
and using the sparsity of the support to imply a high- valued element. Various uncrossing 
techniques in Combinatorial Optimization are very useful in the first step. However, new 
ideas are typically required in the second step which can be usually carried out with a "token 
charging" argument to prove by contradiction the presence of a high value element in the 
support: Assign a set number, k, of tokens to each support variable (now assumed to be low 
valued for contradiction) and redistribute these tokens so as to collect k tokens per tight 
constraints and show some leftover tokens for the contradiction. While there is no unified 
way to arrive at the specific token redistribution argument, some insight can be gained 
by looking at the structure of tight independent constraints and their differences as was 
done in the case of spanning trees, network matrices, submodular flows and the survivable 
network design and STSP problems. Furthermore, these token charging arguments can 
involve integral redistributions or a fractional redistribution based on the values in the 
extreme point; several examples of both types of token redistributions were presented, e.g., 
in the chapters on spanning trees. 

We believe the iterative method can be used to prove integrality of even more gen- 

t © Copyright 2011 by Lap Chi Lau, R. Ravi and Mohit Singh. Published 2011 by Cambridge 
University Press. 
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eral classes of problems. Indeed recent research [8] continues to push the envelope of such 
proofs to such general classes as lattice polyhedra. Other recent results [101, 134] have been 
obtained also using the iterative method and its variants. Some promising areas for future 
investigation include an alternate proof of integrality of totally unimodular matrices start- 
ing from their definition based on the bicoloring characterization due to Ghila-Houri [149], 
as well as exploring the relation of the techniques used in the iterative method to the 
characterization of totally dual integral systems. 

While the iterative method is versatile, it is much less so than the other classical 
methods to prove integrality of polyhedral relaxations such as Total Unimodularity and 
Total Dual Integrality. All the results on exact optimization in this book are re-derivations 
of the results achieved by these two methods. An intriguing open question is to more 
formally derive a relation between the iterative method and these other more powerful 
methods for proving integrality. Nevertheless, a key advantage of the iterative method is 
its adaptability to designing approximation algorithms for the base problems augmented 
with complicating constraints. 

On the approximation algorithms front, the iterative method belongs in the class of 
techniques for designing approximation algorithms based on a linear programming relax- 
ation of the problem, such as deterministic rounding, randomized rounding, the primal-dual 
method, and methods based on Lagrangean relaxations. Like the primal-dual method, this 
method shares a rich intersection with the set of polynomial-time solvable exact character- 
izations that can be proven using the method, as we demonstrated in many of the earlier 
chapters. Looking ahead, the iterative method might offer new insights into the design 
of improved approximation algorithms for the traveling salesperson problems (TSP) and 
its many variants, given their close connection to degree-bounded spanning tree problems. 
Some initial connections in this direction have already been explored [131] but much more 
remains to be done in this direction. 

We close with the hope that our monograph will add the iterative method to the set of 
general techniques in the toolkit for the design of exact and approximation algorithms based 
on linear programming characterizations. We also hope that it will encourage a unified 
presentation of many of these beautiful results at an elementary level for the setting of 
undergraduate classrooms as well as for self-study by inspired enthusiasts of combinatorial 
optimization. 
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